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The paper describes clustering problems from the combinatorial viewpoint. A brief systemic survey 
is presented including the following: (i) basic clustering problems (e.g., classification, clustering, sorting, 
clustering with an order over cluster), (ii) basic approaches to assessment of objects and object prox¬ 
imities (i.e., scales, comparison, aggregation issues), (iii) basic approaches to evaluation of local quality 
characteristics for clusters and total quality characteristics for clustering solutions, (iv) clustering as 
multicriteria optimization problem, (v) generalized modular clustering framework, (vi) basic clustering 
models/methods (e.g., hierarchical clustering, k-means clustering, minimum spanning tree based clus¬ 
tering, clustering as assignment, detection of clisue/quasi-clique based clustering, correlation clustering, 
network communities based clustering), Special attention is targeted to formulation of clustering as mul¬ 
ticriteria optimization models. 

Combinatorial optimization models are used as auxiliary problems (e.g., assignment, partitioning, knap¬ 
sack problem, multiple choice problem, morphological clique problem, searching for consensus/median 
for structures). 

Numerical examples illustrate problem formulations, solving methods, and applications. The material 
can be used as follows: (a) a research survey, (b) a fundamental for designing the structure/architecture 
of composite modular clustering software, (c) a bibliography reference collection, and (d) a tutorial. 

Keywords: Clustering, Classification, Combinatorial optimization. Assignment, Multicriteria problems, 
Decision making. Heuristics, Composite problem frameworks. Applications, Applied artificial intelligence 
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1. Introduction 

Recently clustering/classificataion problems have been widely used in many domains (Table 1.1). 


Table 1.1. Main application domains of clustering-like models/problems 


No. 

Some applied domains 

Basic applied problem(s) 

Source(s) 

1. 

Design/analysis of information 
systems, information retieval 

Document clustering/classification, 
design of hierarchy/ontology/menu, 
information retrieval 

132150119114181 

|453I534I557I593| 

1598161416581 

2. 

Web systems, web services 

Clustering of Web sites, design of 
hierarchical Web systems, search 

16717417518512021 

|418I534I550I652I 

3. 

Data mining&knowledge discovery 

Detection of information objects, 
associations, rules, structures 

15511371199I255I262I 

|265I279I435I446| 

4. 

Medical/technical diagnostics, 
system testing, maintenance of 
systems 

Dehnition of solution classes, 
diagnostics (assignment/multiple 
assignment of patients/system 
components into solution classes), 
clustering of symptoms/faults, etc. 

1214121512821.3411 

|369I394I422I488| 

I565l602l632l66,5i 

5. 

Graph partitioning (network 
design, VLSI design, etc.) 

Grouping of graph vertices 

[191261181121812,541 

132013281,58216251 

6. 

Cell formation in industrial/ 
manufacturing systems 

Grouping of machines 

h22l228l.537l.5,5.5i 

7. 

Anomaly detection (networks, 
distributed systems, etc.) 

Finding patterns in data that do 
not conform to expected behavior 

1931211148716,591 

8. 

Computer vision (images/scenes, 
object trajectories) 

Clustering/segmentation of images, 
shape analysis, detection of events 

[30213101,54215,561 

9. 

Trajectory clustering/classification 
(e.g., air-trafHc control), system 
testing/maintenance, monitoring 

Tracking; traces initialization/ 
maintenance; classification/ 
clustering/fusion of streams 

1541104121312471 

12771360136513891 

|392I399I400| 

10. 

Chemistry, biology, gene expression 
data clustering in DNA microarray 
technology 

Classification/clustering of 
chemical objects (elements, 
structures), detection of natural 
structures, interesting patterns 

[313,512,5211241 
[1 721306141 514431 

1,5211,5221.5,3.5161 li 

161316411626) 

11. 

Communication/sensor systems/ 
networks, computer networks 

Management, clustering of nodes, 
clustering based routing, 
detection of cluster heads, 
design of hierarchical network 

111211,361371 

[38148191198| 
[101120,51221122,3] 

133,51336153816441 

12. 

Management, planning, marketing, 
evaluation of economical objects 
(e.g., financial instruments, tasks, 
firms, countries) 

Hierarchical management, design 
of management hierarchy, team 
design, segmentation of market, 
segmentation of customers 

124149183112011601 

11631243127,312741 

[329137813921442] 

[4631,5141,548160,5) 

13. 

Social sciences, political marketing, 
social network analysis, recognition 
of communities, econometrics, etc. 

Clustering of social/phychological 
objects, analysis of networks/ 
hierarchies, evaluation, planning 

[461176116912,37] 

1266139514.3814701 

14741507159416091 

14. 

Education (evaluation, course 
design, cluster grouping, planning) 

Evaluation (students, courses), 
clustering of students, timetabling, 
educational data mining 

157180111311331 

[220145614991511] 

1512151316161 


The significance of clustering/classification is essentially increased, for example, in the following con¬ 
temporary fields: data analysis, management and decision making, communication systems, engineering, 
chemistry, biomedicine, information retrieval, system monitoring, social sciences, network modeling and 
analysis (e.g., |9l37lH8llh3l224l3()2l43bl443lhl8lh()8lbb7lbbb| l (Fig. 1.1). 

In two recent decades, excellent well-known surveys and books on clustering problems and methods 
have been published (e.g., [30214351623162416^ ). Many research publications including surveys and 
books are targeted to special clustering approaches, for example: clustering based on fuzzy data (e.g.. 
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|144|180|'l. support vector clustering (e.g., m), cross-entropy based clustering (e.g., |312l348j l. online 
clustering (e.g., |43IMI<j2l3b()| l. dynamic clustering (e.g., [481104) 1. consensus clustering (e.g., [1241245) 1. 
graph-based clustering (e.g., [19612361340) 1. clustering ensembles (e.g., [59712241278) 1. clustering based 
on hesitant fuzzy information (e.g., [1101656) 1. multicriteria clustering (e.g., [156) 1. correlation clustering 

(e.g., m)- 



Fig. 1.1. Clustering as basic support problem 

In recent decade, the significance of combinatorial approaches to clustering have been increased. In 
Table 1.2, some research efforts in combinatorial approaches to clustering are pointed out. 

This material contains an author’s “architectural” engineering glance to combinatorial clustering prob¬ 
lems. A special attention is targeted to formulation of clustering problems as multicriteria optimization 
models which are based on usage of various quality parameters for clustering solutions (e.g., clusterings as 
partitions, hierarchical clusterings). Combinatorial optimization models are used as auxiliary problems 
(e.g., partitioning, assignment, knapsack problem, multiple choice problem, matching problem). 

In recent years, various fuzzy clustering methods have been widely studied and used (e.g., |42ll()9lll()l 
I180I238I287I302I330I347I440I467I529I628I656] ! and these approaches are not considered in the material. 

The presented materials can be useful for educational courses and student projects in computer science, 
engineering, management, social sciences applications (e.g., [37713801382) 1. 









































Table 1.2. Combinatorial approaches to clustering 


No. 

Approaches, algorithmic schemes 

Source(s) 

1. 

Some surveys: 


1.1. 

General 

I218I302I437I 

1.2. 

Graph clustering 

|533| 

1.3. 

Approximate graph partitioning 

|189| 

1.4. 

Cross-entropy method for clustering, partitioning 

|348I52()I568| 

1.5. 

Cell formation (in industrial engineering) 

|228I537I555| 

1.6. 

Clustering ensemble algorithms 

m 

1.7. 

Multicriteria classification and sorting methods 

151816661 

2. 

Basic combinatorial optimization problems: 


2.1. 

Minimal spanning tree approach 

I244I445I479I492I582I6()6I626I66()| 

2.2. 

Partitioning based clustering 

I26I67I1341166118915541 

2.3. 

Assignment/location based clustering 

|227| 

2.4. 

Graph matching 

m 

2.5. 

Dominant set based clustering 

(10112631400148916431 

2.6. 

Covering based clustering 

j9l446l525| 

2.7. 

Clique based clustering 

I9l55l8ll176l236l340l539l 

2.8. 

Structural clustering (detection of communities) 

I7I459I461I496I625I 

3. 

Correlation clustering 

(41401159I345I564I 

4. 

Graph-based data clustering with overlaps 

m 

5. 

Segmentation problems 

m 

6. 

Cluster graph modification problems 

(53915401 

7. 

Multi-criteria decision making in clustering-sorting 

12141508150916661 

8. 

Consensus clustering: 


8.1. 

Voting-based consensus of cluster ensembles 

(2815221 

8.2. 

Consensus partitions 

m 

9. 

Algorithmic schemes: 


9.1. 

Enumerative methods: 


9.1.1. 

Branch-and-bound methods 

m 

9.1.2. 

Dynamic programming 

16401 

9.2. 

Local optimization heuristics: 


9.2.1. 

Simulated annealing algorithms 

17814711536| 

9.2.2. 

Tabu search algorithms 

(4711561 1 

9.2.3. 

Ant colonies algorithms 

(31 1 16371 

9.2.4. 

PSO methods 

(1 02(577(592| 

9.2.5. 

Variable neighborhood search 

I268l269l270l271l 

9.3. 

Genetic algorithms, evolutionary strategies 

1301142I288I477I587I 

9.4. 

Hyper-heuristic approach 

(1 29135215861 












































































6 


2. General Preliminary Glance 
2.1. Preliminaries 

From the structured viewpoint, the following basic clustering problem formulations can be pointed 
out |1 5,111 71 1224I2H112751,1001,1021:121 i:t42i:t4:tl4.H4l4.15l44:tl447l5nai51 (i) set partitioning clustering 

(Fig. 2.1), (ii) classification (i.e., solution classes are predefined as in diagnostics) (Fig. 2.2), (iii) 
sorting (group ranking, stratification) problem (the obtained clusters are linear ordered) (Fig. 2.3), (iv) 
hierarchical clustering (Fig. 2.4), (v) multiple clustering (i.e., obtaining n different clustering solutions 
while taking into account n goals/models; alternative clustering) (Fig. 2.5), and (vi) consensus clustering 
(aggregation of clustering solutions; clustering ensembles) (Fig. 2.6). 

Table 2.1 contains initial data for an illustrative numerical example: (a list of item/students and their 
skill estimates upon parameters/criteria): ( 1 ) formal approaches, modeling (i.e., mathematics, physical 
modeling) Ci, ( 2 ) applied computer science, computing (i.e., software development, software implemen¬ 
tation, computing) C 2 , (3) engineering science domain (i.e., information transmission, radio channels, 
antenna devices, sender/receiver devices, networking) C 3 , (4) measurement radio techniques C 4 , and (5) 
preparation of the technical documentations (reports, papers, presentations C 5 . Here ordinal scale [3,4, 5] 
is used: excellent (“5"), good (“4"), sufficient (“3"). 


Clusters 


Initial set 
of elements 
(objects, 
items 



Fig. 2.1. Clustering as partitioning 



Initial set 
of elements 
(objects 
items 



(Class 2 


Predefined 

solution 

classes 



Fig. 2.2. Classification problem 


Initial set 
of elements 
(objects, 
items 


Ranking (linear 
ordered clusters) 



j Layer I 

Step 3 

V 

() Layer 2 

Step 2 

p- 

b Layer (fc — I) 

Step 1 


Layer k 

Step 0 


C 1,2,3,4,5,6 ) 



Fig. 2.3. Sorting (stratification) problem Fig. 2.4. Example of hierarchical clustering 


Clustering solution 1 Clustering solution k Clustering solution n 



OPo 


Modef\ 

Model 

^Model 

(goal) 1 \ 

(goal) k 

/ (goal) 



Initial set of elements (objects, items) 


Fig. 2.5. Illustration for multiple clustering 
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Consensus (e.g., median) of clusterings 




t 


Aggregation of clusterings, summarizing 
(clustering ensembles) 


Clustering 1 Clustering | k 

O O 


Model/ 
method 1 



Clustering 


n 


Model/ 
method k\ 



Initial set of elements (objects, items) 

Fig. 2.6. Illustration for consensus clustering, summarizing 


This numerical example illustrates the pointed out 5 types of clustering problems on the basis of the 
same initial data (Table 2.2, for 12 students): (i) clustering as partitioning (Fig. 2.7a), (ii) classifica¬ 
tion problem (“soft” problem formulation) (Fig. 2.7b), (hi) sorting problem (stratification, multicriteria 
ranking) (Fig. 2.7c), (iv) hierarchical clustering (Fig. 2.8), and (v) consensus clustering (Fig. 2.9). Note, 
consensus clustering is based on three initial clustering solutions: 

(a) five clusters (from partitioning problem, Fig. 2.7a): {6, 9,10}, {1,8, 3}, {2, 5, 7,11}, {12,14}, {3,4}; 

(b) four clusters (from sorting problem. Fig. 2.7c): {6,9}, {1,3,10}, {4, 8,13,14}, {2, 5, 7,11,12}; 

(c) four clusters (from hierarchical clustering. Fig. 2.8): {6,9,10}, {1, 3}, {4,8,13,14}, {2, 5, 7,11,12}. 

In recent decade, the significance of the fifth type of clustering problem has been increased: network 

clustering (e.g., detection of community structures) (Fig. 2.10). Here initial information consists in 
description of network: i.e., set of vertices/nodes and set of edges. Weights of vertices and/or edges can 
be used as well. 


Table 2.2. Items, estimates upon criteria 


Item (student) 

Cl 

C2 

Ca 

C4 

C5 

Student I 

4 

4 

5 

5 

4 

Student 2 

3 

3 

3 

4 

3 

Student 3 

4 

4 

4 

5 

4 

Student 4 

5 

4 

4 

3 

5 

Student 5 

3 

3 

3 

4 

3 

Student 6 

5 

5 

5 

5 

5 

Student 7 

3 

3 

3 

4 

3 

Student 8 

4 

3 

4 

4 

3 

Student 9 

5 

5 

5 

5 

5 

Student 10 

5 

5 

5 

4 

5 

Student 11 

3 

3 

3 

5 

3 

Student 12 

3 

5 

3 

3 

3 

Student 13 

5 

3 

4 

3 

3 

Student 14 

3 

5 

3 

5 

4 





(Cluster 

^ 6r9;io y 

/Ouster^ /Cluster 

^CsTmj v^ jiirM } j 

(" Cluster 3 :^ /Cluster^ 
^ { 2 ,5; rVii }J ^ {3; 4 } J 

(a) clustering as partitioning 


Class 1: Theory, 

En gineering science Comput ing 

p3} |1^ 

1 f b, 3, 6,9,10} X j 
V1^’ 8}V 7/{i4}j y 

]2,5Xri} J 

Class 3: Engineering 
(b) classification (“soft” case) 



(c) clustering as sorting 


Fig. 2.7. Numerical examples of clustering/classification problems 



( ^Cluster 1: {6,9,10}^ Cluster 2: {1,3}^ ( ^Cluster 3: {4,8,13,14}^ Cluster 4: {2, 5, 7,11,12}^ 
Fig. 2.9. Illustrative numerical example of consensus clustering (“median” clustering solution) 



Fig. 2.10. Illustration for network clustering 


A generalized scheme of clustering processes consists of the following [22I182I275I300I302I354I435I5091 
Ifi23lfi24| (Fig. 2.11): 

Phase 1. Collection of initial data on the applied situation. 

Phase 2. Analysis of applied situation(s) and formulation/structuring of clustering problem: 

(2.1.) generation/detection and description of initial objects/elements; 

(2.2.) generation/detection of element parameters (i.e., feature selection or extraction); 

(2.3.) selection/design of proximity measures (for elements, for clusters) and types for inter-cluster 
criterion and for intra-cluster criterion; and 

(2.4.) selection of basic clustering model(s) (i.e., hierarchical clustering, partitioning clustering). 

Phase 3. Selection/design of clustering solving scheme (solving method/procedure). 

Phase 4- Implementation of clustering procedure(s). 

Phase 5. Analysis of clustering solution(s) (i.e., clusters validation). 
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BASIC LAYER: clustering 



SUPPORT LAYER: data, methods, knowledge/experience 




Support information: 

(a) data bases, 

(bl knowledge bases, 

(c) experience of expert (s) 


t t / 


Support procedures: 

(a) clustering methods, 

(bj optimization methods. 


DI optimization u 
2 ) Al-techniques 


Fig. 2.11. System two-layer scheme for clustering 


Fig. 2.12 depicts a structural description for main machine learning paradigms (e.g., |222I313I3431 
1441156316071595] 1: (a) unsupervised learning, (b) reinforcement learning, and (c) supervised learning. 
Basic clustering problems/models corresponds to unsupervised learning. Evidently, in complex cases, it 
is possible to analyze clustering results to correct the clustering solving process (i.e., as in reinforcement 
learning or in supervised learning). 



Fig. 2.12. Three learning paradigms |607| 


2.2. Objects, evaluation, problems 
2.2.1. Examined objects, parameters 

It is reasonable to point out the basic examined objects: 

L item (i.e., element, point), 

2 . set of items (e.g., initial set of items), 

3. subset of items, cluster (e.g., a subset of initial items), 

4 - clustering solution as partitioning of the initial item set into set of clusters, and 

5. clustering solution and an order over its clusters: 

5.1. linear order over the clusters (sorting problem), 

5.2. hierarchy over the clusters, 

5.3. poset over the clusters. 

Significant auxiliary problems consist in measurement of proximity for pair of objects: 1. item and 
item, 2. item and cluster, 3. cluster and cluster, 4- clustering solution and clustering solution, 5. order 
over clusters and order over clusters, and 6 . clustering solution, order over its clusters and clustering 
solution, order over its clusters. 

In recent years, a special research attention is targeted to aggregation of clustering solutions and 
evaluation of the obtained aggregated clustering solution(s). 

Generally, basic problem types, parameters, and criteria are pointed out in Table 2.2 (e.g., I7I6I25I1481 
ll77l229l3()ni3()2l333l375l43()l435l452l458l459l461l5()ll5()2l594l6nil623l624l625l648| L 
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Table 2.2. Basic parameters in clustering/classification problems 


No. 

Parameters, requirements 

Description(s) (e.g., type, evaluation scale) 

I. 

Elements (items/objects): 


1 .1. 

Type of element (s) 

One-type/multi-type element (s), whole 
element (s) or structured/composite element (s) 

1 .2. 

Description of element 

Quantitative, nominal, ordinal estimate(s); 
fuzzy, multiset estimates, vector estimates, 
binary relation(s) over elements 

1.3. 

Proximity for elements pair 

Metric/proximity, ordinal estimate, fuzzy 
estimate, multiset estimate, vector estimate 

1.4. 

Proximity between element and 

Metric/proximity, ordinal estimate, fuzzy 


element set (e.g., cluster) 

estimate, multiset estimate, vector estimate 

1.5. 

Proximity between element set and 

Metric/proximity, ordinal estimate, fuzzy 


element set (e.g., two clusters) 

estimate, multiset estimate, vector estimate 

II. 

Clusters/classes (clustering solution): 


2 .1. 

Definition type for clusters/classes 

1 . Predefined clusters (classification problem) 

2 . Clusters are defend under solving process 
(clustering problem) 

2 .2. 

Constraints for clusters 

Number of clusters, number of cluster elements 

2.3. 

Order over clusters/classes 

Independent clusters, linear order/chain, ranking 


(binary relation(s)) 

(layered structure), hierarchy (e.g., tree), poset 

2.4. 

Clustering solution validity: 

Quality, correspondence to requirements, etc. 

2.4.1 

Basic criteria te.sr.. [6|177|502|fi48j'): 


2.4.1.1 

Compactness (minimization): 

Uniqueness of objects in each cluster: 


(i) intra-cluster “distance”. 

e.g., closeness to cluster centroid in cluster. 


(ii) object positioning. 

maximum distance between objects in each 
cluster, closeness of object to cluster centroid 
“good” correspondence of object to cluster 

2.4.1.2 

Isolation or separability (maximization): 

Well-separated clusters: e.g., maximum 


inter-cluster “distance” 

distances between cluster centroids 

2.4.1.3 

General (maximization): 

(i) number of correctly positioned objects, 

(ii) number of “good” clusters (Fig. 2.13), 



(iii) connectedness. 

similar objects are neighboring 

2.4.2 

Quality of structure over clusters 

Similarity to predefined structure 

2.5. 

Modularity (community structure based 
network clusterine. Fie. 2.14 |7|459|496|') 

Maximization 

III. 

Fuziness/softness of problem/model: 


3.1. 

Hard problem 

Assignment of each item to the only one cluster 

3.2. 

Fuzzy/soft problem 

Assignment of each item to many clusters 

IV. 

Problem time mode: 


4.1. 

Off-line (statical) mode 

Collection of initial data and processing 

4.2. 

On-line mode (dynamics) 

On-line processing (e.g., stream data) 

V. 

Complexity of clustering process: 


5.1. 

Algorithmic complexity 

Estimate 

5.2. 

Volume of required data 

Estimate of required data 

5.3. 

Volume of required expert’s work 

Estimate of expert/expert knowledge 


(22) e 




V 

■‘Good” clusters 

Fig. 2.13. Illustration for “good” clusters Fig. 2.14. Illustration for community structures 
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2.2.2. Basic scales 

Usually, the following main approaches (i.e., scales, types of estimates) for assessment of vector com¬ 
ponent (i.e., Xi,i = l,m) are used (e.g., [2751300130213751392143516231624] 1: (i) quantitative estimate 
(Fig. 2.15a), (ii) ordinal estimate (Fig. 2.15b), (hi) nominal estimate, (iv) poset-like scale (Fig. 2.15c), 
(v) fuzzy estimate, (vi) multiset based scales: (a) multiset estimate for evaluation of composite system 
(Fig. 2.16, [372137413861392] b (b) interval multiset estimate [3861392] . and (vii) vector-like estimate (i.e., 
multicriteria description) (e.g., [3741392] !. Here, traditional fuzzy estimates and hesitant fuzzy estimates 
are not considered (e.g., [110I628I647I656] '). 


Worst 

point 


Best 

point 

(g) » 


t'l t'2 

(a) quantitative scale 


Worst 

point 


K {k — 1) 

(b) ordinal scale 


Best 

point 

2 1 


Worst Best 

points points 

(c) poset-like scale 


< 3,0,0 > Best (ideal) 

I point 

< 2 , 1,0 > 


< 2 , 0,1 > < 1 , 2,0 > 


< 1,1,1 > < 0,3,0 > 


< 1 , 0,2 > < 0 , 2,1 > 


< 0 , 1,2 > 

^ L o Worst 
< 0,0, 3 > point 


Fig. 2.15. Illustration for scales Fig. 2.16. Multiset based scale 


# S = X-kY*Z 

51 = X2 * Fi * Z2 

52 = X3 *¥2 * Z3 


X 




^Xi(2) ^yi(3) ^Zi{l) 
■^ X 2 { 1 ) ^^>2(1) <?)^2(1) 
l^Xsil) ^Z3(3) 

Fig. 2.17. 3-component system 


Some fundamentals of multisets and comparison of sets and multisets have been described in 1 [26013381 
1629) 1. Our brief description of multiset estimates is the following (e.g., [372I374I386I392] !. The approach 
consists in assignment of elements (1,2,3,...) into an ordinal scale [ 1 , 2 ,...,Z]. As a result, a multiset 
based estimate is obtained, where a basis set involves all levels of the ordinal scale: H = { 1 , 2 ,..., Z} (the 
levels are linear ordered: 1 2 ^ 3 ^ ...) and the assessment problem (for each alternative) consists in 

selection of a multiset over set H while taking into account two conditions: 

1 . cardinality of the selected multiset equals a specified 

number of elements ry = 1, 2, 3,... (i.e., multisets of cardinality rj are considered); 

2. “configuration” of the multiset is the following: the selected elements of H cover an interval over 
scale [ 1 ,Z] (i.e., “interval multiset estimate”). 

Thus, an estimate e for an alternative A is (scale [1,Z], position-based form or position form): e(A) = 

(ryi,..., 77t,..., ? 7 /), where rjc corresponds to the number of elements at the level l {l = 1,1), or e(A) = 
m m V3 vi 

{1,..., 1, 2, ...2,3,..., 3,..., I,..., 1}. The number of multisets of cardinality rj, with elements taken from a 
finite set of cardinality I, is called the “multiset coefficient” or “multiset number” 1 [3381629'] 1: = 

i{i+i)(i+2)...(i+7]-i) ^ number corresponds to possible estimates (without taking into account interval 
condition 2). The basic multiset estimate (i.e., without taking into account condition 2) can be used 
as integration of ordinal estimates to obtain a resultant estimate for a composite system (when system 
components are evaluated via ordinal scale [1,2,3]) (e.g., [37213741392) 1 (Fig. 2.16, Fig. 2.17). 

In the case of condition 2 (i.e., interval mutliset estimate), the number of estimates is decreased. 
Generally, assessment problems based on interval multiset estimates can be denoted as follows: 

A poset-like scale of interval multiset estimates for assessment problem is presented in Fig. 2.18. 
Calculation of multiset estimate is be based on transformation of vector ordinal estimate. An illustrative 
numerical example for obtaining multiset vector estimate is the following: 

X = (0, 3,1,0, 2,1) ^ e(x) = (co, Ci, 62 , 63 ) = (2, 2,1,1), 

where equals the number of ordinal estimate k in ordinal vector estimate x. 
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(4,0,0) 

(3,1,0) 

( 2 , 2 , 0 ) 

(1,3,0) 

^ ( 2 , 1 , 1 ) 

(0,4,0) 

^ ( 1 , 2 , 1 ) 

(0,3,1) 


C3D (i>i>2) 


1 2 3 





( eV ) (0,2,2) 

1 2 

3 



( ) (0,1,3) 

1 2 

1 1 

3 



( ) (0,0,4) 

8 . Scale, estimates (P^d) [3§$l392j 

1 2 
Fig. 

3 

2.1 


Brief descriptions of the scales above and their transformations (e.g., mapping, integration) are pre¬ 
sented in |386I390I392) . The basic types of operations over estimates above are the following (Table 2.3) 
(e.g., 139013921 ): 

1 . transformation of an estimate (including transformation into an estimate of another type), 

2 . calculation the difference (distance/proximity) for two estimates, 

3. integrations (e.g., summarization, average estimate or median-like estimate). 

A basic approach to integration of qualitative estimates (including vector qualitative estimates) con¬ 
sists in summarization or calculation of an average value. On the other hand, integration of of ordinal 
estimates, poset-like estimates, and multiset estimates (i.e., “structural” estimates) is usually considered 
as calculation of a median-like estimate (i.e., as agreement/consensus structure). This kind of problems 
is formulated as an optimization (or multicriteria optimization) (e.g., |372I384I392] ). Sometimes, the re¬ 
sultant integrated estimate can be considered as a “fuzzy” structure (e.g., interval-like integrated ranking 
for integration of rankings in [3721392) ). 

The above-mentioned operations (integration as summarization, calculation of proximity, calculation 
of a median estimate) for multiset estimates are presented in [3861392) . 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


1 2 3 


p3,4 
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Table 2.3. Basic operations over estimate(s) 


No. 

Type of initial 
estimate(s) 

Transformation 
of estimate 
(resultant estimate) 

Difference (proximity) 
of two estimates 
(resultant estimate) 

Integration 
of several estimates 
(resultant estimate) 

1. 

Quantitative 

1. Qualitative 

2. Ordinal 

1. Quantitative 

2. Ordinal 

1. Quantitative 
(average value) 

2. Vector estimate 

2. 

Ordinal 

1. Ordinal 

1. Ordinal 

1. Ordinal 

(average / median) 

2. Vector estimate 

3. Multiset estimate 

3. 

Nominal 

1. Nominal 

1. Ordinal 

1.Vector estimate 

4. 

Vector 

1. Vector 

2. Ordinal 

3. Multiset 

1. Qualitative 

2. Ordinal 

3. Vector 

1. Vector 

(average / median) 

4. 

Poset-like 

1. Poset-like 

2. Vector 

3. Multiset 

4. Ordinal 

1. Qualitative 

2. Ordinal 

3. Vector 

1. Poset-like 
(average / median) 

2. Multiset 

5. 

Multiset 

1. Multiset 

2. Ordinal 

1. Ordinal 

2. Multiset 

3. Vector 

1. Multiset 
(average / median) 


2.2.3. Assessment/evaluation problems 

Generally, the following measurement approaches are under examination (for solutions as clustering, 
ranking, consensus clustering, and their components): 1. metrics/proximities: (i) for objects/clusters, 
(ii) for orders over clusters, (iii) for clustering solutions; 2. total measures for clustering solution(s) (total 
quality); 3. measure for an aggregation structure (i.e., median, consensus, agreement structure, covering 
structure). Table 2.4 involves a list of basic types of the assessment/evaluation problems. 

2.2.4. Calculation of objects/clusters proximities 

The initial set of items (objects) under examination is A = {ai,...,aj, 
eters oi x G A and vector estimate x = {xi, ...,xi, ...,Xm) (Fig. 2.19). 
metrics/proximities for objects/clusters are considered: 

Case 1. objects/points - object/point (Fig. 2.20, Fig. 2.21a). 

Case 2. object - cluster (subset) (Fig. 2.21b). 

Case 3. Intra-cluster proximity (distance): for all elements in cluster. 

Case 4-. Inter-cluster proximity (distance): cluster - cluster (Fig. 2.21c) 

Further, the above-mentioned cases are examined. 

Case 1. Metrics/proximity for two objects (points) (Fig. 2.20, Fig. 2.21a). 

Two objects are examined while taking into account m parameters (criteria): (i) the first object 
X G A\ vector estimate x = (xi, ...,Xi, ...,Xm), (ii) the second object y G A: vector estimate y = 
{yi ,..., i/i,..., ym)- Here, Xi, yi {i = 1, m) are real numbers. The basic types of proximities/distances (as an 
integration of vector-like difference between vector element estimate upon quantitative scales) between two 
objects X and y {D{x,y)) are the following (e.g., [231127513001302132113431392143414351490162316241666] '): 

1. Euclidean distance: D{x,y) = [ 1^* “ 2/*^ 

2. Minkowski distance: D™'^^{x,y) = [ \xi — yiY {r > 0). 

3. Manhattan distance: D^°‘^^{x,y) = YYILi \xi — Vi\- 

4- Tchebyschev distance: D‘^^‘^^{x,y) = maxjg{i_..._m} \xi — yi\. 

5. Canberra distance: Il°“™(x, 2 /) = YllYi ^ ® 2/i > 0)- 

Vector proximity: D{x,y) = {\xi - yi\, ...,\xi - yi\,—Yxm - ym\); 

7. Ordinal estimate (for example, scale [0,1, 2,3,4]: 0 corresponds to the same objects or equivalent 
ones, 1 corresponds to the “very close” objects, 2 corresponds to the “close” objects, 3 corresponds to 


There are m param- 
Here, the following cases of 
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the “different” objects, 4 corresponds to the “very different” objects). 


Table 2.4. Assessment/evaluation problems 


No. 

Analyzed object (s) 

Goal 

Approach 

Estimate type (scales) 

1 . 

Element (item) 

Description 

Assessment 
(e.g., expert, 
statistics) 

Quantitative, ordinal, 
nominal, fuzzy, multiset, 
vector 

2 . 

Two elements/items 

Proximity/distance 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

3. 

Element, 
cluster (subset) 

Proximity/distance 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

4. 

Two clusters 
(subsets) 

Proximity/distance 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

5. 

All elements of 
cluster (subset) 

Intra-cluster proximity 
(quality of cluster as 
element proximity) 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

6 . 

All elements in all 
clusters (clustering 
solution) 

Total intra-cluster 
proximity (generalized 
element proximity 
in all clusters) 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

7. 

All clusters of 
clustering solution 

Inter-cluster proximity 
(quality of solution as 
integrated proximity 
between clusters) 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

8 . 

Clustering solution 

Criteria, requirements/ 
constraints 

Constraint 

satisfaction/ 

optimization 

Quantitative, ordinal, 
fuzzy, multiset, vector 

9. 

Two rankings (two 
sorting solutions) 

Proximity/distance 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

10 . 

Ranking (sorting) 
solution 

Proximity to standard 
solution, correspon¬ 
dence to requirements 
(quality of solution) 

Constraint 

satisfaction/ 

optimization 

problem 

Quantitative, ordinal, 
fuzzy, multiset, vector 

11 . 

Two hierarchies (e.g., 
trees) over clusters 

Proximity/distance 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

12 . 

Hierarchy over 
clusters (solution) 

Proximity to standard 
solution, correspon¬ 
dence to requirements 

Constraint 

satisfaction/ 

optimization 

Quantitative, ordinal, 
fuzzy, multiset, vector 

13. 

Aggregated item(s): 
“median item/set”, 
“center”, covering 
object (e.g., ellipsoid) 

Quality of aggregated 
item(s) (value, 
vector) 

Calculation 

Quantitative, ordinal, 
fuzzy, multiset, vector 

14. 

Consensus clustering 
(median/agreement) 

Quality of consensus 
solution (value, 
vector) 

Constraint 

satisfation/ 

optimization 

Quantitative, ordinal, 
fuzzy, multiset, vector 

15. 

Consensus ranking/ 
sorting solution 

Quality of consensus 
solution (value, 
vector) 

Constraint 

satisfaction/ 

optimization 

Quantitative, ordinal, 
fuzzy, multiset, vector 

16. 

Consensus hierarchi¬ 
cal clustering 
solution 

Quality of consensus 
solution 

Constraint 

satisfaction/ 

optimization 

Quantitative, ordinal, 
fuzzy, multiset, vector 


8 . Fuzzy estimate of object proximity and/or vector fuzzy estimate of object proximity. Fuzzy esti¬ 
mates are widely used in clustering methods (e.g., |42ll09ll80l238l287l302l330l347l440l467l529) f including 
clustering based on hesitant fuzzy estimates (e.g., [11016281656] j (here they are not considered). 
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9. Multiset estimate of object proximity [3861392) . 

10. Angular separation (e.g., [490] . Fig. 2.20): 11“"®“'“’'(x,y) = ^ 

iZ^i=i “i Z^i=i 

The similarity measure corresponds to the angle between the item vectors in directions of x and y |490j . 


• . • . \ Set of 

_ / obiects/ 

a : a; = (cci, ...,a:i, ...jCCm) items 
® • / . 

/ A 


Fig. 2.19. Illustration for object/item 



Objects 
set A 


Fig. 2.20. Angle proximity 


Objects set A Objects set A Objects set A 



(a) object x, object y (b) object x, cluster Y (c) cluster A, cluster Y 
Fig. 2.21. Object/cluster proximities 


Illustrative numerical examples are the following. 


Example 2.1. An ordinal vector proximity can be used for calculation of the proximity of two ordinal 
vectors x and y (ordinal scale [1, 5] for vector estimates, ordinal scale [0,4] for vector proximity): 

(a) ordinal vector estimates are: x : x = (3,4,1,1, 2, 5), y : y = (3,1, 2,1,4,4); 

(b) ordinal proximities are: 

(i) D{x,y) = (xi -yi,X 2 -y 2 ,X 3 -y3,X4 -yi,X3 -yb.XQ - ye) = (0,3,-1,0,-2,1), 

(ii) D'{x,y) = {\xi - yi|, |x 2 - y 2 |, |x3 - ysl, ^4 - y4|, |x5 - ysl, \xe - yel) = (0,3,1,0,2,1). 


Example 2.2. An ordinal vector proximity can be used for calculation of the proximity of two 
quantitative vectors x and y (quantitative scale (0,5) for vector estimates, ordinal scale [0,4] for vector 
proximity). Two initial quantitative vector estimates are: (a) x = (0.3,3.5,1.4,1.5, 2.3,4.9), (b) y = 
(0.3,0.8, 2.2,1.6, 3.9,4.1). Examples for calculation of proximities are: 

(1) The quantitative proximity is: 

D{x,y) = (jxi -yi|,|x 2 -y 2 |,|a ;3 -y3|,|x4 - y 4|, |a^5 - V bV W - y&\) = (0.0,2.7,0.8,0.1,1.6,0.8). 

(2) Calculation of the vector ordinal proximity D'(x, y) = (di, d 2 , ds, d 4 , ds, de) can be based on rule: 


di 


0, if 0.0 <\xi - yi\ < 0.2, 
1, if 0.2 < \xi - yi\ < 0.5, 

< 2, if 0.5 < \xi — yi\ < 0.8, 

3, if 0.8 < \xi — yi\ < 3.5, 

4, if 3.5 < \xi — yi\ < 5.0. 


The resultant vector ordinal proximity is (based on D{x, y) ): D'{x, y) = (0, 3,2,0, 3, 2). 


Example 2.3. In the case when quantitative vector estimates are transformed into ordinal vector 
estimates, the example 1 is obtained. 


Case 2. Object-cluster (subset) (Fig. 2.21b). The following initial information is considered: 

(i) object X € A: x = (xi, ...,Xi, ...,Xm) (vector estimate); 

(ii) elements of cluster Y = {y^, ...,y‘', ...,y^} C A, y^\ y^ = (y^,..., ..., y^) (vector estimate) 

(Vy^ G Y). 

Proximity D{x,Y) between object x and cluster Y can be considered, for example, as the following: 
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(1) =minvj,cey D{x,yi)-, 

(2) D™“-(x,r) = maxvj,cey D{x,yi)-, 

(3) D-'’{x,Y) = l/y Ev.cey D{x,y^); 

(4) Y) = D{x, y), where y is centroid (or median point) of cluster Y. 

Clearly, various measurement approaches (i.e., metric/proximity) described for case 1 can be used. 

Example 2.4. Let us consider item x: x = (0.3,3.5,1.4,1.5) and cluster Y: y^ = (1.1,4.0, 3.2,4.3), 
(2.0,5.!,2.5,5.2),^= (1.3,4.7,4.2,1.6). 

For four case above, the following proximities are obtained (Euclidean distance of two items is used): 

(1) D™‘^{x, Y) = minvyCgy D{x, y^) = min{£)(a;, y^), D{x, y^), D{x, y^)) = min{3.4,4.5,3.3)} = 3.3; 

(2) D'^°‘^{x,Y) = maxvyCgy D{x,y^)= ma,x{D{x,y^), D(x,y^), D{x,y^)) = max{3.4,4.5, 3.3)} = 4.5; 

(3) D-^x,Y) = i Evycev Dix,y^)= ^iDix,y^) + Dix,y^) + D{x,y^)) = l/3(3.4 + 4.5 + 3.3) = 3.7; 

( 4 ) ^“"‘(a;, Y) = D{x, y) = £>((0.3,3.5,1.4,1.5), (1.16,1.1,1.9, 2.2)) = 2.7. 

In the case of vector proximity D{x,y‘’) the following calculation schemes can be used: 

Scheme 1. Preliminary transformation of vector proximities D{x,y‘^) (Vy^) into ordinal vector prox¬ 
imity and usage of the measurement methods above (e.g., proximity to the closest point of cluster); 

Scheme 2. Integration of proximities D{x,y‘’) (Vy*”) to obtain a general vector proximity D{x,Y) and 
transformation the general vector proximity into a number (quantitative or ordinal). 

Example 2.5. Numerical examples for the above-mentioned two cases (i) and (ii) are the following: 

Schem^l: i:>(a;,y'^) = (|a;i - y^|, |x 2 _^ y^l, ^3 - y||, 1 x 4 - y^l). _ 

Thus, Dix,y^) = (0.8,0.5,1.8, 2.8), D(x,y2) = (1.7,1.6,1.1,3.7), D{x,y^) = (1.0,1.2, 2.8,0.1). 

Transformed vector proximities (into ordinal vector estimates, rules from example 2) are: 

Wix, yi) = (2,1,3, 3), Wix, y2) = (3,3, 3,4), S^(x, y^) = (3, 3,3, 0). 

Evidently, point y^ can be considered as the closest point for item x (version of proximity between 
point X and cluster Y as proximity of point x and the closest point of the cluster Y), i.e., D'{x,Y) = 
;^(x,yi) = (2,l,3,3). _ 

Scheme 2: Let us integrated general vector proximity is: D{x, Y) = D'{x, Y) = (2,1,3, 3) (from scheme 
1 above). A transformation process of the ordinal vector into an ordinal estimate can be based on the 
following simplihed rule (for ordinal vector (/3i, ...,/3i, ...,13m))' 

'0, if 0< Ei=T;7i/5* 

1, if m< Y.i=T-^Pi <‘^rn, 
a'= < 2, if 2m < < 3m, 

3, if 3m < < 4m, 

. 4, if 4m < ■ 

Here, m = 4 and the resultant ordinal estimate for D'{x,Y) is: 2. In general, it may be reasonable to 
use a multiset estimate, for example, e{x, Y) = (0,1,1, 2) because in D'{x, Y) = (2,1,3, 3): 0 estimates 
at the level 0, 1 estimate at the level 1, 1 estimate at the level 2, 2 estimate at the level 3. 

Case 3. Intra-cluster proximity (distance) (for all element pair in cluster). The following initial 
information is considered: elements of cluster Y = {y^,..., y^,..., y**} C A, y^: y4 = (y^,..., yf,..., y£) 
(vector estimate) (Vy*” G Y). 

Intra-cluster proximity for cluster Y: I{Y), Y Q A can be considered, for example, as the following: 

(1) minimum distance 7*”‘’'“’™“(y) = min^^y^Il(y^,y^); 

(2) maximum distance £"*'’“’™“'^(E) = max^^y^^^y^ D{y^,y^); 

(3) average distance = 1 D(yf,y«). 

Various measurement approaches (i.e., metric/proximity) from case I can be used as well. 

Example 2.6. Let us consider cluster Y = {y^,y^,y^} (from example 2.4). 

Euclidean distances between cluster elements are: £)(y^, y^) = 1.8, £>(y^, y^) = 3.0, Zl(y^, y^) = 4.0. 

Thus, versions of intra-cluster proximities for cluster Y are: 
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jintra,min ^ g jintra,max ^ q ji7itra,av ^Y^'j _ 2 Q 

On the other hand, element distances 

D{y\y^) = (0.9,1.1,0.7,0.9), My\y^) = (0.2, 0.7,1.0, 2.7), Diy^, y^) = (0.7, 0.4,1.7,3.6) 
can be transformed into ordinal vector proximities (by rule above) 

^{y\y^) = (3,3,2,3), W, y') = (0,2,3,3), Wiy^, y^) = (2,1,3,4). 

After transformation of ordinal vector proximities into integrated ordinal estimates, the following es¬ 


timates are obtained (by rule above): D'{y^,y^) => 2, D'{y^,y^) 1, D'{y^,y^) 


2. Here, 


minimum ordinal intra-cluster proximity equals 1, maximum ordinal intra-cluster proximity equals 2. 

Transformation of ordinal vector proximities into multiset estimates leads to the following estimates: 
e(y\y^) = (0,1,1,3,0), e(y\?/3) = (1,0,1,2,0 ) e(^^j /^) = (0,1,1,1,1). Note, the average multiset 

estimate can be computer as a median estimate |386I392) . 

Case 4. Inter-cluster proximity (distance) (for two clusters. Fig. 2.21c). The following initial informa¬ 
tion is considered: (i) elements of cluster AT = {x^, C A, x^: x^ = {x\, ...,x^, ...,x^) (vector 

estimate) (Va;^ G X); (ii) elements of cluster F = {y^,..., ..., y''} C A, y^\ y? = (y^,..., y^,..., y^) (vec¬ 
tor estimate) (Vy‘’ G F). In the main, the following basic types of inter-cluster distances are considered: 
(a) minimum element proximity (distance) (“single link”), (b) maximum element proximity (distance) 
(“complete link”), (c) average element proximity (distance), (d) median element proximity (distance), 
(e) proximity (distance) between cluster centroids. For example, inter-cluster proximity for two clusters 
X,Y: F), Ai, F C A can be considered as the following: 

(1) /™*'=”(F,F) =minj^^_^^^ D{x^,y^)- 

(2) J“‘™"(X,F) =max^^^^^^ 7?(a^«,y^); 

( 3 ) ^ 

Clearly, various measurement approaches (i.e., metric/proximity) described for case 1 can be used. In 
the case of vector proximity D{x^,y‘’), the following calculation scheme can be used: 

(a) preliminary transformation of vector proximity D{x^,y^) {\/x^,y‘’) into a number (quantitative or 
ordinal) and usage of the above-mentioned measurement methods; 

(b) integration of proximities D{x^,y^) {\/x^,y^) to obtain a general vector proximity D{X,Y) and 
transformation the general vector proximity into a number (quantitative or ordinal). 

Example 2.7. Let us consider cluster F = {y^,y^,y^} (from example 2.4) and cluster and cluster X: 
x^ = (0.1, 1.O, 0.2, 0.3), = (0.3,0.9,0.5,0.6). Distances between elements of X and F are presented in 

Table 2.5 {D{x^, y^)). Table 2.6 (Euclidean distance D(x^,y^)), and Table 2.7 (ordinal distance D'{x^,y^), 
by rule above). 

Table 2.5. Vector distance D{x^,y^) 



y' 


7 

x‘^ 

(1.0,3.0,3.0,4.0) 
(0.8,3.1,2.7,3.7) 

(1.9,4.1,2.3,4.9) 
(1.7,4.2,2.0,4.6) 

(1.2,3.7,4.0,1.3) 
(1.0,3.8,3.7,1.0) 


Table 2.6. Euclidean distance D(x^,y^) 



y 

y 


X^ 

6.0 

7.0 

5.8 

x^ 

5.7 

6.8 

5.4 


Table 2.7. Ordinal vector distance D'{x^,y^) 



¥ 

7 

7 

X^ 

(3.3.3.4) 

(2.3.3.4) 

(3,4,3,4) 
(3,4,3,4) 

(3,4,4,3) 
(3,4,4,3) 


Thus, versions of inter-cluster proximities for clusters Ai and F are: 

jinter,min ^ A jinter,max _ y q jinter^av _ 0 ^ 

Results of transformation of ordinal vector proximities into integrated ordinal estimates (by rule above) 
are presented in Table 2.8. Here, minimum ordinal inter-cluster proximity equals 2, maximum ordinal 
inter-cluster proximity equals 3. Resultant multiset estimates are presented in Table 2.9. 
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Table 2.8. Integrated ordinal estimates 




^ 1 

,.3 


y 

y 

y 


3 

3 

3 


2 

3 

3 


Table 2.9. Resultant multiset estimates e{x^,y‘') 



y" 

y^ 



(0,0,0,3,1) 
( 0 , 0 , 1 , 2 , 1 ) 

( 0 , 0 , 0 , 2 , 2 ) 
( 0 , 0 , 0 , 2 , 2 ) 

( 0 , 0 , 0 , 2 , 2 ) 
( 0 , 0 , 0 , 2 , 2 ) 


2.2.5. Quality of clustering solution 

Here “hard” clustering problem is examined. Consider initial items/elements of element set A = 
{oi, (Fig. 2.19). Two types of initial information for clustering can be examined: 1. there 

are m parameters/criteria and measurement of a is based on vector estimate x = {xi ,..., xi ,..., Xm) (Fig. 
2.19); 2. binary relation(s) over element set A (including weighted binary relation(s); this is a structure 
over obtained clusters of a graph). Note, the first type of initial information can be transformed into the 
second type. A clustering solution consists of the following two parts: 

(1) Clusters X = {Xi, ...,Xt, ...X^}, i.e. dividing set A into clusters: X^ C A Vr = 1, A; = \X^\ is 
the cluster size (i.e., cardinality for cluster X^, l = 1, A). 

(2) Structure over clusters (if needed). Let F(X) be a structure over the clusters of the clustering 
solution X, i.e., there exists digraph G = X,r(X). Let r(Xj be the structure over the elements of 
cluster Xt (VX^ e X ). 

The list of basic quality characteristics is the following (Table 2.10): 


Table 2.10. List of quality characteristics 


No. 

Quality type 

Notation 

Description 

1. 

Cluster 

X, 

1 < i < A 

1 . 1 . 

Intra-cluster distance 

j“*™(xo 

Proximity between elements of cluster 

1 . 2 . 

Size of cluster 


Number of elements in cluster Xt 

1.3. 

Quality of cluster form 


Closeness to predefined form 
(e.g., ball, ellipsoid) (if needed) 

1.4. 

Size of cluster region 


Difference between “max” and “min” 
coordinates (by parameters) 

1.5. 

Quality of cluster content/structure 


Configuration of element types (if needed) 

1 . 6 . 

Quality of cluster structure 


Proximity of structure over cluster 
elements to predefined structure (if needed 

11. 

Clustering solution 

X 

X = {Xi,...,X„...,X;,} 

2 . 1 . 

Total intra-cluster quality 

Q“*™(X) 

Integration of intra-cluster parameters 
(/“*™(X,), byi = T;A) 

2 . 2 . 

Total inter-cluster quality 

Q™*®”(X) 

Integration of inter-cluster parameters 

(/™*-(X,,,X,J, by 61 , 62 , H^t 2 ) 

2.3. 

Number of clusters (A) 

Q”"™(X) 

Number of clusters in clustering solution 

2.4. 

Closeness to cluster size 

Q^“'(X) 

Balance by cluster size, 

closeness to predefined balance vector 

2.5. 

Quality by forms of clusters 

Qform^X^ 

Integration of cluster form parameters 

2 . 6 . 

Parameter of cluster regions 

Q”®9(X) 

Integration of cluster regions sizes 
(by coordinates) 

2.7. 

“Correlation clustering functional” 

Qcorr(^) 

Integration of maximum agreement 
(in each cluster) and minimum 
disagreements fbetween clusters! [39140) 

2 . 8 . 

Quality of modularity 

Q™°‘'(X) 

Parameter of network modularitv |226|459| 

III. 

Quality of structure over clusters 

Q®*™“(X) 

Closeness to predefined structure 

IV. 

Multicriteria quality 

Qix) 

Integrated vector of quality (e.g., Q{X) = 
(Q™*™(X), Q“‘'=”(X), Q^'^^iX), Q''^s(X) ) 
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1. Quality of clusters (i.e., local quality parameters in clustering solution): 

1.1. Intra-cluster distance (i.e., general proximity between elements in each cluster): 

(t = t;a). 

Version 1. Quantitative parameter as integration of quantitative element proximities (distances) in the 
cluster (this is described in previous section, case 3). 

Version 2. Multiset parameter as integration of ordinal estimates of element proximities. The approach 
is illustrated by example. 

Example 2.8. Example for three clusters is depicted in Fig. 2.22: Xi = {1,2, 3,4}, X 2 = {5,6,7}, 
Xz = {8, 9,10,11,12}. Ordinal scale [1, 2, 3] for estimates of element similarity is used: 

1 corresponds to “very similar”, 

2 corresponds to “medium level”, 

3 corresponds to “very different” (in this case the edge in Fig. 2.22 is absent). 

Ordinal proximities of edges are presented in Table 2.11. The resultant multiset intra-cluster parameters 
for clusters are: /“‘™(Xi) = (2,3,1), r”‘™(X 2 ) = (1,1,1), r”‘™(X 3 ) = (4, 2,4). 


Cluster Xi Cluster X 2 Cluster X^ 



Fig. 2.22. Local intra-cluster quality (for cluster) 


Table 2.11. Ordinal proximities (intra-cluster, edge ( 11 , 12 )) 


ii 

i2 : 2 3 4 6 7 9 10 11 12 

1 

2 1 2 

2 

3 2 

3 

1 

5 

2 3 

6 

1 

8 

112 3 

9 

3 3 1 

10 

3 2 

11 

1 


1.2. Number of elements in cluster (or in each cluster, i.e., cluster size) corresponds to constraints, for 

example: n~ < = \X^\ < 7r+ (7r“,7r+ are predefined limits of the cluster size) {VX^ G X). 

The quality parameter corresponds to external requirement (from the viewpoint of applied problem(s), 
e.g., teams, communication systems). 

1.3. Quality of cluster form (e.g., body, envelope, cover), for example: sphere/bail, ellipsoid, globe 
(i.e., closeness to the required cluster form). 

1.4. Quality as constraint for size of cluster region (limits for interval for coordinates of cluster 

elements). Let us consider cluster X = {x ^,..., ..., x'^}, parameter estimates of each cluster element 

are (vector estimate, parameters i = l,m): x^ = (x^, ...,xf, ...,x|j). Constraints are (by each parameter 
Vi = T70) (Fig. 2.23): 

I max x^ — min xf| < di, Vi = l,m. 

5 = 1,0 5 = 1,0 

The quality parameter corresponds to external requirement (from the viewpoint of applied problem(s), 
e.g., communication systems). 
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1.5. Quality of the cluster contents/structure (if needed), for example (a composite “team”): 1 element 
of the 1st type, 3 elements of the 2nd type, 2 elements of the 3rd type, 1 element of the 4th type. Here 
proximity of the obtained cluster content to the required content can be considered. 

1.6. Quality of cluster structure (if needed) for cluster ( 'iX^ S X ), i.e., proximity (5(r(Xi), r°(XQ), 
where r°(Xt) is the predefined structure over the cluster elements. 

2. Total quality for clustering solution (i.e., for cluster set): 

2.1. Total intra-cluster quality for clustering solution Q™‘”“(X) is an integrated measure of intra¬ 
cluster parameters (/™‘’'“(XQ) of all clusters in clustering solution (i.e., i = 1, A). 

Version 1. Total qualitative quality parameter for qualitative local estimates: 

t=T7A 

Note, integration process can be based on summarization and some other operations (maximization, 
minimization, etc.). 

Version 2. Total multiset quality parameter for multiset local estimates. The approach is illustrated 
by example. 

Example 2.9. Example for three clusters is depicted in Fig. 2.24 (for simplification the cardinality 
of clusters is the same): Xi = {1,2,3}, X 2 = {4,5,6}, X^ = {7,8,9}; clustering solution is: X = 
{Xi^X 2 ,X^}. Ordinal scale [1,2,3] for estimates of element similarity is used: 1 corresponds to “very 
similar”, 2 corresponds to “medium level”, 3 corresponds to “very different” (in this case the corresponding 
edge is absent). Ordinal proximities of edges in clusters are presented in Table 2.12. 

The resultant multiset intra-cluster parameters for clusters are: 

/“‘™(Xi) = (1,2,0), /“‘™(Jf2) = (2,1,0), /“‘™(X3) = (2,1,0). 

These multiset estimates correspond to poset (lattice) from Fig. 2.16. 

Integration of the above-mentioned intra-cluster multiset estimates can be based on two methods (e.g., 
[3861392) !: 

(a) summarization (by the vector components): (5“‘’'“(X) = (5,4,0), the obtained integrate estimate 
corresponds to an extended lattice (not to lattice from Fig. 2.16); 

(b) searching for a median multiset estimate: (3™‘”“(Af) = (2,1,0) (lattice from Fig. 2.16). 



Fig. 2.23. Size of cluster region 


Cluster X 2 



Fig. 2.24. Intra- and inter-cluster qualities 


Table 2.12. Ordinal proximities (intra-cluster, edge ( 11 , 12 )) 



12 : 2 3 5 6 8 9 

1 

2 2 

2 

1 

4 

2 1 

5 

1 

7 

2 1 

8 

1 
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2.2. Total inter-cluster quality for clustering solution is an integrated measure of inter¬ 
cluster parameters of all cluster pairs in clustering solution (i.e., ii = 1,A, t 2 = 1,A, 

Version 1. Total qualitative quality parameter for qualitative local estimates as integration of all 
qualitative two-cluster inter-cluster proximities/distances (from previous section, case 4): 

Q^nter ^ E )' 

Li — l,X,L2 — l,X,Li^L2 

Note, integration process can be based on summarization and some other operations (maximization, 
minimization, etc.). 

Version 2. Total multiset quality parameter for multiset local estimates. The approach is illustrated 
by example. 


Example 2.10. The example is based on data from previous example 2.9 (i.e.. Fig. 2.24). Table 2.13 
contains inter-cluster ordinal proximities. 

Table 2.13. Ordinal proximities (inter-cluster, edge {i,j)) 


i 

j ■ 

4 

5 

6 

7 

00 

9 

1 


3 

3 

3 

3 

3 

3 

2 


2 

2 

3 

3 

3 

3 

CO 


3 

3 

3 

3 

3 

2 

4 





3 

3 

3 

5 





3 

3 

3 

6 





2 

3 

2 


Inter-cluster multiset estimates are: 

pnter^Xi,X 2 ) = (0,2,7), (Xi,X 3 ) = (0,1,8), T"*""(X 2 ,X 3 ) = (0,2,7). 

Integration of the above-mentioned intra-cluster multiset estimates can be based on two methods (e.g., 
|38bl392| b 

(a) summarization (by the vector components): Q™*®’'(X) = (0,5,22); 

(b) searching for a median multiset estimate: (5*"*®’'(X) = (0, 2, 7). 

2.3. Total number of clusters in clustering solution Q”“’”(X), for example: T“ < A(X) < T+ 

(T“, T+ are predefined limits of the total cluster number). The quality parameter corresponds to external 
requirement (from the viewpoint of applied problem(s)). This is connected to 1.2 

2.4. Closeness of element cluster sizes in clustering solution to the predefined cluster size constraints, 

i.e., balance (or imbalance) of cluster cardinalities for example: 7 r~ < |XJ| < tt’*' ( 7 r“, 7 r+ are 

general limits of each cluster size. Evidently, here the balance/imbalance (i.e., out-of-balance) estimate of 
a clustering solution can be consider as the number of clusters that corresponds (or does not correspond) 
to the constraints. The estimates can be examined as a vector-like estimate or a multiset estimate, 
for example: the number of “good” clusters (with “good/right” cluster size), the number of quasi-good 
clusters (with quasi-right cluster size), and the number of other clusters. This parameter corresponds to 
external requirement (from the viewpoint of applied problem(s)). Now let us describe the version of the 
vector-like estimate. The notations are as follows: (a) 7 r°(X) is the number of clusters in X in which 
the cluster size complies with the predefined limits, (b) 7 r“''*(X) is the number of clusters in X where 
the cluster size X^ more then (upper limit) by I elements, (c) 7 r“*(X) be the number of clusters in 
X where the cluster size X^ less then tt~ (bottom limit) by I elements. As a result, the following vector 
estimate can be considered: 

Q''“'(X) = (V(X),...,^-i(X),^0(X),7ri(X),...,7r'^(X)). 

Note, a close type of the vector estimate (vector proximity) has been suggested for comparison of 
rankings in |372j . The approach is illustrated by example. 
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Example 2.11. Initial set of objects is: A = {1,2,3,4, 5, 6 , 7, 8,9,10,11,12,13,14,15,16,17}, cluster¬ 
ing solution is: X: Xi = {1, 5, 7), Xa = {2}, X 3 = (3, 6,10,13,17}, X 4 = (11,12}, X 5 = {4,12,14,15}, 
Xq = {8,16}. The following constrains for cluster size are considered: ifi = 2, jfa = 3. Vector estimate 
for balance of cluster cardinalities is: Q*'“*(X) = ( 7 r“^(X), 7 r°(X), 7 r^(X),..., 7 r^(X)) = (1,3,1,1). 


The considered approach is close to T-balanced partitioning (clustering solution X) when size of each 


obtained cluster jX^j 


T(X) 


(VXt e X) where T(X) (i.e., A) is the number of obtained clusters. 


2.5. Total quality for balance (or imbalance) of cluster forms (i.e., cluster bodies/covers) in a clustering 
solution ( 5 '^°’’™(X), for example: majority of clusters of a clustering solution have the same (or about 
the same) bodies (e.g., spheres/balls, ellipsoids, globes). 

Evidently, it is possible to consider a measure of imbalance, analogically as in parameter 2.3. 


2.6. Total quality Q’"®®(X) as constraints for size of cluster regions (limits for interval of cluster 
element coordinates for each cluster). Let us consider cluster X^ = {x''’^,..., x'-’^, Parameter 

estimates of each cluster element are (vector estimate, parameters i = l,m and clusters t = 1,A): 
= {x^^, ...jX^’^, ...,x‘A). Constraints are (by each parameter Vi = (Fig. 2.23): 

I max x^ — min xf | < dj, \/i = l,m {each coordinate/parameter), Vt = l,A {each cluster). 

5 = 1 . 0 , 5 = 1 . 0 , 

The quality parameter corresponds to external requirement (from the viewpoint of applied problem(s), 
e.g., communication systems). 


2.7. The “correlation clustering functional” to maximize the intra-cluster agreement (attraction) and 
the inter-cluster disagreement (repulsion) has been proposed in 139140] (Q“’'’’(X)). Here, partitioning 
a fully connected labeled graph is examined (label “-I-” corresponds to edge between similar vertices, 
label corresponds to edge between different vertices). The optimization functional Q‘^°'^^{X) is an 
integration (i.e., summarization) of two components: (i) the maximizing number of edges between 
clusters (i.e., minimizing disagreements), (b) the number of “-I-” edges insides the clusters (i.e., maxi¬ 
mizing agreements) (e.g., |4l33l39l40ll59l345l564l662] i. Weighted versions of the “correlation clustering 
functional” are considered as well (e.g., |94I95I159] L 

2.8. Modularity of clustering solution Q^°^{X) is dehned as follows (e.g., |226l457l459l4m] i (Fig. 
2.25). Let G = {A, E) be an initial graph, where A is the set of nodes, E is the set of edges. Clustering 
solution for graph G is: X = (Xi,..., X,,..., X^}. Let A’' be the set of nodes in cluster X, (i = 1, A ). Let 
E'- be the set of internal edges in cluster X, (t = 1,A ), i.e., all corresponding nodes belong to A'-. Let 
E'- be the set of external edges for cluster X, (t = 1, A ), i.e., the only one corresponding node belong to 
A'-. The definitions are illustrated in Fig. 2.25 for a four cluster solution (cluster X 3 ). 

Thus, the following parameters for each cluster X, can be used: 

(a) e, = { % edges in module t ), 

(b) o, = ^ ( % edges with at least one end in module l ). 

Further, general modularity of clustering solution for graph G is: 

A 

Q-^(X)=^ (e,-(a,)^). 

t=i 

Clustering problem to maximize the modularity is NP-hard [73] . The approach is illustrated by example. 


Example 2.12. Let us consider modularity parameters for clustering solution from Fig. 2.25. Here, 
\E\ =26. Parameters for clusters are: 

(1) \E^\ = 6 , \E^\ = 4, ei = 0.23, ai = 0.38; (2) \E'^\ = 4, \E'^\ = 4, 62 = 0.15, 02 = 0.3; 

(3) \E^\ = 3, \E^\ = 4, 63 = 0.115, 03 = 0.27; (4) = 4, = 2, 64 = 0.15, 04 = 0.23. 
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The resultant modularity parameter for clustering solution is: = (0.23 — 0.14) + (0.15 — 

0.09) + (0.115 - 0.073) + (0.15 - 0.053) = 0.09 + 0.06 + 0.42 + 0.097 = 0.667. 

Internal edges 



Fig. 2.25. Modularity in graph clustering 

3. Quality of structure over clusters (e.g., tree, hierarchy) (if needed) ( ). Here 

a proximity of the obtained structure r(X) in clustering solution X and a predefined structure F^ is 
examined: = (5(r(X),r°). Clearly, various scales for assessment of the proximities can be 

used (e.g., qualitative, ordinal, vector-like, multiset) (e.g., |384I392) 1. 

4. Generally, it is reasonable to consider multicriteria quality of clustering solutions that integrates 

the above-mentioned clustering characteristics, for example: QiX) = {X),Q™*^°‘{X),Tr{X)). 


Evidently, general integrated vector-like estimate for assessment of total quality of clustering solution 
can be examined, for example: Q{X) = ((5(X), 5(r(X), r°). As a result, the clustering problem can be 
formulated as generalized multicriteria optimization problem (i.e., Pareto-efficient solutions have to be 
searched for), for example: 

min g“*™(A), max Q™*"gA), s.t. Q'’“'(A) ^ 7r°, = (5(r(A), r°) < (5°. 

In the case of multiset estimates, the multiple criteria optimization clustering problem can be considered 
on the basis of quality lattices (poset-like scales) as follows (i.e., Pareto-efRcient solutions over posets have 
to be searched for): 

min g“‘’'“(A), (by lattice, Fig. 2.26a) 
max g*"*®’'(A), (by lattice, Fig. 2.26b) 

s.t. ^ 7°, Vi = 1, A, 7° is reference multiset estimate (by lattice. Fig. 2.26c), 

Q^°‘\X) :< 7r°, 7r° is reference multiset estimate (balance by cluster size, by lattice. Fig. 2.267), 

Qstruc _ ^(r(x),r°) < (5°, (closeness to predefined general structure P®). 


Thus, Fig. 2.26 illustrates the integrated “discrete space” (poset) for 
of clustering solution X. 


total multiset based vector quality 




(c) local intra-cluster 
quality 7“‘™(A,),Vi 
Fig. 2.26. Illustration for quality posets (lattices) 



(d) total balance by^ 
cluster sizes Q^°'\X) 
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In the case of “soft” clustering problems, it is necessary to examine measures of solution “softness” 
(e.g., total parameter for intersection of clusters). 


2.2.6. Comparison of two clustering solutions 

Comparing methods for clustering solutions (e.g., partitions, hierarchical clustering solutions) have been 
studied many years (e.g., |31l51l207l293l294l428l434l501l661) b The main methods are the following: (a) 
pair counting methods (i.e., how likely two solutions group an elements pair together, or, separate them in 
different clusters); (b) set matching, (c) variation of information. Here, the following kinds of clustering 
solution are considered: 

(1) basic clustering solution X as a set of clusters X = {Xi,X^,X\} (i.e., partition of initial 
elements A = {1, ..., i ,..., n}); 

(2) clustering and order over the cluster set {.^i, ■■■,X^, ...,Xa}: 

(2.1) clustering and linear order over the cluster set, i.e, ranking i?, 

(2.2) hierarchy over the cluster set H =< H,X >, where H is hierarchy over clusters of basic clustering 
solution (partition of elements) X. 

Evidently, general graph over the cluster set can be examined as well. 

Generally, two approaches can be examined for comparison of two clustering solutions: 

(a) difference by structures (“structural” comparison), e.g., a cost of transformation: 

^ F 2 , Ri ^ i?2, Hi ^ H 2 ). 

(b) difference by quality criterion (or criteria), for example: 

Q'(Fi) - Q'(X2), Q"(i?i) - Q"{R2), - Q"'{H2)- 

Here “structural” comparison is considered. The following comparison cases are examined: 

Case 1. Proximity of two clustering solutions Xi and X 2 (Fig. 2.27): D{Xi^X 2 ). 

Case 2. Proximity of two rankings and i ?2 (Fig. 2.28): D{Ri,R 2 ). 

Case 3. Proximity (auxiliary ) of two hierarchies Hi and H 2 (over clusters. Fig. 2.29): D{Hi,H 2 ). 

Case A. Proximity of two hierarchical clusterings Hi and H 2 (e.g., vector; Fig. 2.30): 

D{Hi,H 2) = {D{Hi,H2),D{Xi,X2)). 

Fig. 2.31 depicts an illustrative numerical example for 7 elements: A = {1, 2,3,4, 5,6, 7} (“hard” 
clustering problems). 


Clustering solution Xi Clustering solution X 2 



Fig. 2.27. Proximity for two clustering solutions 


Ranking i?i 


Ranking R 2 


I 


D{Ri, R2) 


I 


I 


I 


Fig. 2.28. Proximity of rankings 


D 

3 


3 

3 


Hierarchy Hi Hierarchy H 2 

D{Hi,H2) 




Fig. 2.29. Proximity of hierarchies 


D{Hi,H 2) = {D{Hi,H2),DiXi,X2)) 



Fig. 2.30. Proximity of hierarchical clusterings 


Further, some simplified versions of comparing methods for clusterings solutions are described (for 
“hard” clustering). 

For case 1 (proximity for clustering solutions, i.e., partitions) the following main approaches are consid¬ 
ered (e.g., [4415111531293129414281433143414361501] '): Misclassification Error distance, distance, Ham¬ 
ming distance, Rand index, Mirkin metric, ordered sets, consensus. Generally, it is possible to use 
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vector and multiset estimates as well. Let us consider a simplified edit proximity as number of steps for 
transformation (i.e., cost of transformation): 

D{Xi => X 2 ): Xi = 1 ^X 2 = {^ 2 . 1 ,^ 2 .aJ. 

The value of proximity will be based on ordinal interval [0,n] (i.e., the number of relocated elements 
in Xi). The following simple algorithm (heuristic) can be used (i.e., Xi ^ X[ = X 2 )' 

Stage 1. Definition: D{Xi => X 2 ) = 0. 

If Ai < A 2 then extension of solution Xi by (A 2 — Ai) empty clusters. 

Stage 2. Calculation of the number of common elements for clusters of the clustering solutions (the 
number corresponds to cluster proximity) ii = l,Ai and ^ 2 , 12 ) '■2 = 1)A2 as intersection (as 

cardinality of the same elements set). As a result, the intersection matrix will be obtained: M(Xi, X 2 ) = 
111^(^1,11, A 2 , 12 ) 11 , 6i = 1,Ai, i2 = l,A2. 

Stage 3. Finding the maximum element in matrix M: 

fJ^r^ax = ^(^X 1 ^ 1,1 ,X2^i") = m ax _ {/i(Ai,ti, A2,t2)}- 


Stage 4- Selection the cluster Xi^^i in solution Ai as cluster of new (transformed from Ai) solution 
A'l). 

Stage 5. Relocation for set Ai,t/ the following elements: 6 € A 2 ,t"| 6 GA 2 ,t" (with deletion of the 
elements from other clusters of solution Ai). 

Increasing D(Xi A 2 ) by the number of the relocated elements. 

Stage 6. Deletion of cluster Ai,t/ and A 2 ,t" from the examination and recalculation of matrix M (i.e., 
deletion of the corresponding line and column). 

Stage 1. If matrix M is empty (i.e., resultant transformed cluster X'l is built) then GO TO Stage 9. 
Stage 8. GO TO stage 3. 

Stage 9. Stop. 


A 

fTA 

2 

3 

4 

5 

6 


(a) elements 



(b) clusterings (c) rankings 


(d) hierarchies over clusters 


Fig. 2.31. Illustrative numerical example 


Example 2.13. The usage of algorithm above for example from Fig. 2.31b is the following. 

Two clustering solutions are under examination: 

(i) Ai = {Ai,i,Ai,2,Ai,3,Ai,4}, Ai,i = {1,2}, Ai,2 = {3,4}, A 1.3 = {5,6}, Ai,4 = {7}, 

(ii) Ai = {Ai,i,Ai,2,Ai,3,Ai,4}, A 2 ,i = {!}, A 2,2 = {2,3}, A2,3 = {4}, A 2.4 = {5,6,7} ). 

Table 2.14 presents the number of common elements for cluster pairs. Fig. 2.32 depicts step-by-step 
building the clustering solution A{ = A 2 . Thus, D(Ai => Ai) = 3 (three elements have been re-located: 
7, 2, 4). 


Table 2.14. Common elements for clusters of solutions Ai, Ai 



1—I 

II 

ci 

^2.2 = {2,3} 

^2.3 = {4} 

A2.4 = {5,6,7} 

^1,1 = {1,2} 

1 

1 

0 

0 

^1.2 = {3,4} 

0 

1 

1 

0 

^1.3 = {5,6} 

0 

0 

0 

2 

^1.4 = {7} 

0 

0 

0 

1 
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Fig. 2.32. Steps for solution Xi ^ = X 2 


For case 2 (proximity for rankings), the following approaches are often considered (e.g., [185118613261 
188418921488] 1 : 1. Kendall tan distance [827] . 2. distances for partial rankings [411190] . and 3. vector-like 
proximity [872] . 

Further, proximity measure as a transformation cost will be used: i? 2 ) = D{Ri R 2 ) (be., the 

number of element re-allocations while taking into account linear ordering over clusters) (this is similar 
to Kendall tau distance). Let a{i,Ri) be the number of layer of element i € A in ranking Ri and 
a{i, R 2 ) be the number of layer of element i £ A in ranking i? 2 - Thus, re-location parameter for i £ A 
is: A(i,i?i R 2 ) = a(i,i? 2 ) — a{i,Ri). Generally, the following proximity is obtained (an analogue of 
Kendall tau distance, ordinal scale [0, (n — l)n]): 

D{Ri ^ R2) = '^ 2 , -^^2)1 = ^ |a(i,i?2) - Q!(i,-Ri)|- 

For vector-like proximity (e.g., [37213841392] 1 . the following parameter is considered: { n € [—{n — 

1), (n — 1)]) which equals the number of elements with A{i,Ri R 2 ) = k. Then, integrated vector-like 
proximity is: D{Ri ^ R 2 ) = (/3-(”-i),...,/S"!, ^ 0 ,/3\ ...,/3(”-i)). 

Example 2.14. Example of two rankings from Fig. 2.31 is considered. Table 2.15 contains numbers 
of re-locations for each element of ranking i?i to obtain ranking i? 2 . 

Table 2.15. Re-location of elements for transformation i?i R 2 


i G A 

Re-location 

A(i, Ri R 2 ) 

1 

0 

0 

2 

1 

1 

3 

1 

1 

4 

0 

0 

5 

0 

0 

6 

1 

-2 

7 

1 

-1 


Thus, the resultant transformation cost (the number of re-location) is: D{Ri R 2 ) = 5. For vector¬ 
like proximity [87218841892] . the following result is obtained: D{Ri R 2 ) = (0,0, 0,0,1,1, 3, 2, 0,0, 0,0,0). 

For case 3 (proximity for hierarchies, e.g., trees), the following main approaches are in use (e.g., 
[69I218I384I392] '): 1. metrics/distances (e.g., alignment distance, top-down distance, bottom-up distance) 
(e.g., [805I419I571I591] !. 2. tree edit distances (i.e., correction algorithms) (e.g., [100I526I569I572] !. 3. 
largest common subtree, median tree or tree agreement/consensus (e.g., [12I20I605] !. and 4- vector 
proximity (e.g., [8841892] '). For the simplification, the following correction algorithm will be considered 
as the number of addition and deletion of edges/arcs to transform initial tree (hierarchy) Hi into the 
resultant hierarchy (tree) H 2 '. D{Hi => H 2 ). For example in Fig. 2.31d, D{Hi ^ H 2 ) = 1 (i.e., deletion 
of the only one arc). 

In case 4, it may be reasonable to use an integrated vector-like resultant including two main components: 
proximity for clustering solution and proximity for hierarchies. For example in Fig. 2.31d, the following 
initial information is examined: Hi =< Xi^Hi > and H 2 =< X 2 ,H 2 >■ Thus, the following resultant 
two-component proximity is obtained: D{Hi ^ H 2 ) = {D{Xi X 2 ), D{Hi ^ H 2 ) = (2,1). 
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2.2.7. Aggregation of objects/clustering solutions 

The aggregation problem for N initial objects is depicted in Fig. 2.33 (e.g., 13841392) 1: 

{S'!,..., Se ,..., Sn} => 5“®®. 

The basic types of aggregation problems are listed in Table 2.16. Two notes can be pointed out: 

1. There are aggregation problems in which the resultant (agreement) structure has another type than 
the initial structures, for example: (a) trees are aggregated by graph, (b) trees are aggregated by forest, 
(c) rankings are aggregated by fuzzy ranking, (d) rankings are aggregated by poset (e.g., [971259135^ '). 

2. Separation of two-object aggregation problems is based some situations when these problems can 
have more simple level of algorithmic complexity than many object aggregation problems. 


([ Aggregated object S"*®® 

t 


Aggregation process 




is: 


ix: 


Initial object 5'i'^ Initial object 

Initial object 

Fig. 2.33. Aggregation of objects 


Table 2.16. Types of aggregation problems 


No. 

Type of objects 

Result 

Description 

Some) 

source(s) 

1. 

Two elements (i.e., two 
parameters vectors) 

1.Aggregated element 

2.Integration (cluster) 

1. Median 

1.Integration 


2. 

N elements (i.e., N 
parameters vectors) 

1.Aggregated element 

2.Integration (cluster) 

1. Centroid/median 

2.Integration 

130013021 

3. 

1 element and cluster X 

Extended cluster 

Addition 


4. 

2 clusters Xi, X 2 , 

1. Aggreement cluster 

2. Aggregated cluster 

1. Median / agreement / 
consensus 

2.Integration 


5 . 

N clusters {Xi, ...,Xn} 

1. Agreement cluster 

2. Aggregated cluster 

1. Median / agreement / 
consensus 

2.Integration 


6. 

2 clustering solutions 
^ 1 , ^2 

Aggregated clustering 
solution A “33 

Median / agreement / 

consensus 


7. 

N clustering solutions 

Aggregated clustering 

Median/agreement / 

I28I44I107I264I293I 


{Xi,...,Xn} 

solution A“33 

consensus 

)443I447I560I579| 

8. 

2 rankings Ri, R 2 

Aggregated ranking 

Ragg 

Median/agreement / 
consensus 


9. 

N rankings {i?i,..., i?Ar} 

Aggregated ranking 

Ragg 

Median, agreement/ 
consensus 

1471135113615741 

10. 

2 hierarchies (trees) 

Hi, H 2 

Aggregated hierarchy 
(tree) 

Median / agreement / 
consensus 


11. 

N hierarchies (trees) 
{Hi,...,Hn} 

Aggregated hierarchy 
(tree) H^^g 

Median/agreement / 
consensus 

112120119216031 

12. 

2 hierarchical clustering 
solutions Hi, H 2 

Aggregated hierar¬ 
chical clustering 
solution i7“33 

Median/agreement / 
consensus (by cluste¬ 
ring, by hierarchy) 


13. 

N hierarchical clustering 
solutions {Hi, ..., Hf^} 

Aggregated hierar¬ 
chical clustering 
solution H°-33 

Median/agreement / 
consensus (by cluste¬ 
ring, by hierarchy) 
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The aggregation problems are formulated as a calculation procedure (e.g., calculation of a median 
point) or as an optimization problem to find a set/structure which is median/agreement/consensus for 
the set of initial structures: S'®®® = argminys X]e=Tlv ^{^6 ^)- ^he other hand, the aggregation 

problem can be considered on the basis of optimization approach: 

Find an aggregation object (set, structure) S“®® for the initial set of objects {Si,..., Se,..., Sat} to 
obtain maximum/minimum for quality estimates of S“®® while taking into account requirements (as 
some constraints, e.g., transformation costs) for S“®®. 

Here, the problem is [D^ is a limit for transformation cost): 
max Q(S“®®) s.t. DiSe ^ 

The basic cases for aggregation of N objects are the following: 

Case 1. Aggregation of N objects (i.e., points/clusters). Here, calculations of an average object/centroid 
or a covering object are usually used. 

Case 2. Aggregation of N clustering solutions: (Ai, Xg,X n} => A“®®. 

Case 3. Aggregation of N rankings: {i?i, ...,Re, ...,Rn} => 

Here the following three methods can be used: (i) median consensus method based on assignment 
problem (e.g., |135I136] 1. (ii) heuristic approach (e.g., [471574] . and (iii) method based on multiple choice 
problem (e.g., [372] 1. 

Case 4- Aggregation of N hierarchies (trees): {Hi,Hg,H n} iJ“®®. 

The following methods are often considered for trees: (1) maximum common subtree (e.g., |12jl. (2) 
median/agreement tree (e.g., [20ll92j l. (3) compatible tree (e.g., |261| 1. and (4) maximum agreement 
forest (e.g., |97I259| 1. 

Case 5. Aggregation of N hierarchical clustering solutions: [Hi ,..., Hg ,..., Hn} => i7“®®. 

Here, a composition of case 2 and case 4 can be considered. This aggregation problem is very prospective 
for future study. 

Mainly (e.g., cases 2, 3, 4, 5), the aggregation problems above belong to class of NP-hard problems 
(e.g., [135l261j l. A simplified illustrative numerical example for case 2 is as follows. 

Example 2.15. Three initial clustering solutions for set A = (1,2, 3,4, 5, 6 , 7} are examined: 

(i) Ai = {An,Ai2,Ai3,Ai4}, An = {1,2}, A 12 = {3,4}, A 13 = {5,6}, Aw = {7} (Fig. 2.31b); 

(ii) A2 = {A 2 i,A 22 ,A 23 ,A 24 }, A21 = {!}, A22 = { 2 , 3 }, A23 = { 4 }, A24 = { 5 , 6 , 7 } (Fig. 2 . 31 b); 

(iii) A3 = {A31, A32, A33}, A31 = { 1 , 2 }, A32 = {3,4}, A33 = {5,6,7}. 

The following aggregation problem is under examination (with constraint for cluster size): 

A®®® = arg min D{Xg A). 

In our problem (A = 3), the number of admissible partitions (i.e., clustering solutions) equals 90 
(Cg X Cf). For the simplified calculation, the following admissible aggregated clustering solution is 

—--agg — --o,gg —--(^99 — -'Cl99 

considered: A' = {A'^ ,A '2 jA'g }. Numbers of common elements for clusters of initial clustering 

^ ^ -^-^agg 

solutions Ai, A 2 , A 3 and considered aggregated solution A' are presented in Table 2.17, Table 2.18, 
Table 2.19. The transformation costs are (i.e., numbers of re-allocations): 

-OQQ - CLQQ -—^aOQ 

D{Xi ^ A' ) = 1 , D{X2 => X' ) = 2 , ^(Aa ^ A' ) = 0 . 
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^ --^agg 

Table 2.17. Common elements for clusters of solutions Xi, X' 



X[J^ = { 1 , 2 } 

II 

X™|s = {5,6,7} 

^ 1.1 = { 1 , 2 } 

2 

0 

0 

^ 1.2 = {3,4} 

0 

2 

0 

^ 1,3 = {5,6} 

0 

0 

2 

^ 1.4 = {7} 

0 

0 

1 


^ --^agg 

Table 2.18. Common elements for clusters of solutions X 2 , X' 



X[J<^ = {1,2} 

II 

X[J^ = {5,6,7} 

^ 2,1 = {1} 

1 

0 

0 

^ 2.2 = {2,3} 

1 

1 

0 

^ 2.3 = {4} 

0 

1 

0 

X2.4 = {5,6,7} 

0 

0 

3 


Table 2.19. Common elements for clusters of solutions X 3 , X' 



II 

= {3,4} 

^IT = {5,6,7} 

^ 3,1 = { 1 , 2 } 

2 

0 

0 

^ 3.2 = {3,4} 

0 

2 

0 

^ 3.3 = {5,6, 7} 

0 

0 

3 


2.3. Basic clustering models and general framework 
2.3.1. Basic clustering problems/models 

Table 2.20 contains the list of basic types of well-known clustering problems/models: (e.g., [25122113^ 
llKbll79l2l8l231l27.^l3()()l3()2l321l328l342l34KI43KI4bll.^()9l.^33lb94lb23lb24| i: 

(1) connectivity models (e.g., hierarchical clustering), 

(2) centroid models (e.g., k-means algorithms, i.e. exclusive clustering), 

(3) distribution models (based on statistical distribution), 

(4) subspace models (e.g., bi-clustering or two-mode clustering while taking into account elements and 
attributes), 

(5) graph-based models (e.g., detection of cliques or quasi-cliques/community structures, graph parti¬ 
tioning), etc. 

Note, clustering procedures based on combinatorial optimization problems and/or their composition 
are widely studied |27I267I300I302I437| : 

(i) spanning trees based clustering (e.g., |235l244l445l479l492l606l626l660) i: 

(ii) assignment/location problems based clustering (e.g., [2271323] 1: 

(hi) set covering problem based clustering (e.g., [914461525] !: 

(iv) partitioning problem based clustering (e.g., [13411661189132815541625 ] ) including correlation clus¬ 
tering (e.g., [411591345] !: 

(v) dominant sets/dominating sets based clustering (e.g., [1011263140014891615] !: and 

(vi) clique/community based clustering (e.g., [9I55I170I176I236I340I426I458I459I461I496I535] !. 

Important contemporary clustering problems are targeted to clustering of complex (e.g., composite, 

modular, structured) objects, for example: 

(a) words/chains/sequence clustering (e.g., in bioinformatics) [188] . 

(b) trajectory clustering [21313651400] . 

(c) data stream clustering [27712471389] . 

(d) subspace clustering [913451446] . and 

(e) clustering of structured objects (e.g., trees, graph-based models) [201142115341558] . 

On the other hand, clustering is widely used in complex combinatorial optimization problems, for 
example: 

(1) clustering/partitioning of an initial problem for decreasing the problem dimension, 

(2) clustering as local auxiliary problem(s) (e.g., [332I356I372I392] !. 

















































30 


Table 2.20. Basic types of clustering problems/models 


No. 

Model type 

Some sources 

I. 

Basic problem formulations 


1.1. 

Connectivity models (hierarchical/agglomerative clustering 

I146l275l232l355l5()9l468l63()l 

1.2. 

Centroid models (k-means algorithms, exclusive clustering) 

Il6ll276l3()()l3()2l299l3l8l4351 

1.3. 

Distribution models (based on statistical distribution) 

13001301130214351 

1.4. 

Subspace models (e.g., bi-clustering or two-mode clustering 
while taking into account elements and attributes) 

I300I302I345I415I 

1.5. 

Pattern-based clustering 

|17I345I476I491I604| 

1.6. 

Combinatorial optimization models in clustering: 

1111121814371 


(i) minimal spanning tree based clustering, 

I302I235I244I445I479I492I606I 

|626|660| 


(ii) partitioning based clustering. 

113411661189I328I437I 


(iii) correlation clustering, 

[41401159I345I564I 


(iv) detection of communities structures (clique, etc.), 

I2l8l170l302l459l46ll496l588l 


(v) assigment/location based clustering. 

[22712921 

1.7. 

Overlapping clustering 

m 

1.8. 

Modularity clustering 

[7I73I459I461I625I 

1.9. 

Support vector clustering 

[53111814011 

1.10. 

Spectral clustering models 

[1611254131 71414| 

1.11. 

Symbolic approach in clustering 

I58l63ll65l233l234l364l530l 

1.12. 

Al-based clustering (e.g., knowledge bases, heuristics, 

1141301781142134614531536) 


evolutionary approaches) 

[54415611587) 

1.13. 

Clustering based on Variable Neighborhood Search 

[268126912711 

1.14. 

Neural networks based clustering 

[1861216)3151 

1.15. 

Robust clustering 

1149118812121246131 Oj 

1.16. 

Clustering of structured objects 

|252|558| 

II. 

Fuzzy (soft) clustering problems/models 


2.1. 

Fuzzy clustering 

[42I238I287I302I347I440I 

|4b7|49(,l| 

2.2. 

Fuzzy k-means clustering 

jl38ll80l287l297| 

2.3. 

Kernel-based fuzzy clustering 

1109111812381529) 

2.4. 

Fuzzy clustering for symbolic data/categorical data 

I184I290I330I 

2.5. 

Clustering based on hesitant fuzzy information 

[108111016281656) 

III. 

Stochastic clustering 


3.1. 

Probabilistic clustering 

[621721302134714121570) 

3.2. 

Probability-based graph partitioning, Markov random works 

16711 81 13581410I553I566I 

3.3. 

Cross-entropy method for clustering 

11 551348131 2I520I568I567) 

IV. 

Dynamic clustering, online clustering, restructuring 


4.1. 

Dynamic clustering 

[4818411041164131616441652) 

4.2. 

Dynamic fuzzy clustering 

[144146414801528) 

4.3. 

Online clustering 

I43l54l92l654| 

4.4. 

Restructuring in clustering (i.e., changing of clustering) 

)381|392]. this paper 

4.5. 

Multistage clustering, cluster trajectories 

this paper 

V. 

Very large clustering problems/models 


5.1. 

Clustering of large data sets 

[55I543I653I667) 

5.2. 

K-means clustering for large data sets 

j289l29ll299) 

VI. 

Multiple clustering, framework-based clustering 


6.1. 

Multiple clustering, cluster ensembles, aggregation 

I3I28I153I224I245I278I443) 


clustering, consensus clustering 

|447I560I597I637) 

6.2. 

Unified frameworks, parallel clustering, hybrid strategies 

I146I298I452I468I477I527I657I 


2.3.2. Systems problems 

Generally, the following vital clnstering system problems can be pointed out (e.g., [5211961300130214551 
I540I539I552I577I594I65()0 (Table 2.21). 
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Table 2.21. Basic systems problems in clustering 


No. 

Systems problem 

Some source(s) 

1. 

Formulation/structuring of clustering problem(s) 

I275I3()()I3()2I433I434I435I594| 

2. 

Comparison of models/methods/techniques 

14271501155716501 

3. 

Selection/design of model/method/technique 

|117|209|541| 

4. 

Evaluation of clustering solution(s) 

I4l33l39l40l94l95l159l 

I226I345I457I459I564I662| 

5. 

Validation of clustering solution 

1561177I257I258I359I424I452I 

6. 

Stability of clustering solution 

1511359154115521 

7. 

Robustness of clustering solution 

141109114911881212124613101 

8. 

Cluster editing, cluster graph modification, 

I64I65I66I14711 5712511 


transformation of clustering solution 

1500153915401 

9. 

Identification/selection/assignment of cluster heads 

|82I98I10311 1 9I280I307I577I583I 


(e.g., sensor networks, mobile networks, target tracking) 

i589|638| 

10. 

Prospective clustering problems/approaches: 


10.1. 

Online clustering, clustering data streams 

|43I54I92I247I360I389| 

10.2. 

Multiple clustering, consensus clustering 

|3I28115312241245127814431 

10.3. 

Hybrid (by methods, by data types) clustering methods 

|1 1 513031314I357I404I416I576I584| 


composite, multistage/multilevel clustering methods 
(including adaptation modes) 

15621617161816421 

10.4. 

Expert knowledge based clustering/classification 
(including expert judgment based clustering) 

11 2511 2612141215I453I490I 

10.5. 

Multicriteria optimization clustering 

|156I197I368I392I551I666| 

10.6. 

Clustering with multi-type elements (each cluster is 
composed by compatible elements of different types) 

this paper 

10.7. 

Clustering with hesitant fuzzy sets data 

I627I628I108I 

10.8. 

Fast clustering methods 

|218I170I302I457I459I461| 


In addition, the following system stage can be used: 

“Modification of clustering process (if needed), for example, by the following ways:” (a) modification 
of type(s) of element description(s), (b) modification of element parameters/features, (c) modification of 
element estimates types (or scales), (d) modification of criteria for inter-cluster proximity and intra-cluster 
proximity, (e) modification of criteria for quality of clustering solution(s), (f) modihcation of clustering 
method(s), and (g) searching for additional expert(s). 

2.3.3. General framework 

Fig. 2.6 depicts an example of a clustering framework for multiple clustering. Another generalized 
clustering/classification framework (the viewpoint of a simplified information processing morphology ) is 
presented in Fig. 2.34 (an extension of framework in Fig. 2.11): 

Stage 1. Collection of initial data. 

Stage 2. Analysis of applied situation, problem structuring/formulation: (2.1) selection/generation of 
features/parameters/criteria, (2.2) definition and description (assessment) of the set of objects/items, 
(2.3) selection/design of basic clustering model(s). 

Stage 3. Preliminary data processing: (3.1) calculation of element proximities (distances), (3.2) def¬ 
inition of very close elements (i.e., definition of a small element proximity/distance), (3.3) definition of 
basic relation(s) over element set (a basic relation graph(s)), (3.4) revelation of basic preliminary groups 
of interconnected elements (i.e., some preliminary kernels of clusters). 

Stage 4- Basic clustering: (4.1) selection/definition of basic groups of interconnected elements (i.e., 
some candidates-clusters), (4.2) definition of the basic cluster set, (4.3.) extension of the basic cluster set 
(within framework of feedback). 

Stage 5. Classihcation (if needed): (5.1) assignment of elements into clusters, (5.2) multiple assignment 
of elements into clusters. 

Stage 6. Aggregation of cluster solutions (i.e., consensus clustering/clustering ensemble) (if needed). 
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Stage 7. Analysis of clustering/classification results (clustering solution(s)) (e.g., cluster validity). 


Here, a special support layer can include the following: 1. additional data, 2. expert(s) and expert(s) 
knowledge, 3. additional models (e.g., vertex covering problem, assignment/matching problems, multiple 
criteria sorting/ranking problems, clique/quasi clique problem(s), multiple clique/quasi clique problem(s), 
median/consensus/agreement problems). 


STAGE 1 STAGE 2 STAGE 3 STAGE 4 STAGE 5 STAGE 6 STAGE 7 



Eig. 2.34. General framework for clustering/classification 

Note, the following can be used as alternative morphological components: 

(i) various problem analysis and formulation approaches (e.g., selection of well-known problem state- 
ment(s), design of a new problem formulation(s)), (ii) various element metrics/proximities, (hi) various 
cluster metrics/proximities, (iv) various item assessment techniques (e.g., usage of statistical data, usage 
of expert-based techniques), (v) various clustering methods, (v) various clustering solution aggregation 
methods. 

2.3.4. Example of solving morphological scheme 

A simplified example of morphological scheme for clustering process presented in Fig. 2.34 (an analogue 
of composite strategy for multicriteria ranking/sorting problem [3871392] 1 is the following (Fig. 2.35): 

0. Compressed solving framework S = H-kP-kM-kG-kQ : 

1 . Analysis of situation, problem statement and structuring (i.e., parameters/criteria, scales, etc.), 
assessment H = X (stage 3): 

(1.1) problem formulation X: classification (“hard”) Xi, classification (“soft”) X 2 , clustering (i.e., 
partitioning) (“hard”) X 3 , clustering (i.e., partitioning) (“soft”) X 4 , sorting (“hard”) X 5 , sorting (“soft”) 
Xq, a composite problem Xy. 

( 1 . 2 ) assessment of objects/items Y: usage of statistical data Yi, expert based procedures Y 2 , statistical 
data and expert based procedures Y 3 = Yi&y' 2 - 

2. Criteria/proximities and preliminary processing P = U -kV (stage 3): 

(2.1) proximity/metric for element pair (i.e., similarity measure) U: Euclidean distance (T 2 ) Ui, 
ordinal estimate C/ 2 , multicriteria estimate t/3, interval multiset estimate C/ 4 , 

(2.2) intra-cluster quality (criterion for intra-cluster “distance”, to minimize) V: maximum of element 
pair proximity (single link) Vi, maximum of all element pair proximities (all links or average link) V 2 ; 

(2.3) criterion for inter-cluster “distance” (to maximize) W: minumum “distance” between clusters 
Wi, average “distance” between clusters W 2 . 
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3. Clustering method/model M (stage 4): hierarchical clustering Mi, if-means clustering M2, span¬ 
ning tree based clustering M3, graph method based on detection of cliques/quasi-cliques M4, correlation 
clustering M5, composite method (parallel processing) Mg = M2&M3&M4, composite method (parallel 
processing) M7 = M2&M4&M5. 

4. Aggregation of clustering solutions G (stage 6): none Gi, median-based solving process G2, 
extension of common clustering solution part (i.e., a solution kernel) G3. 

5. Analysis of resultant clustering solution(s) (i.e., cluster validity) Q (stage 7): none Qi, expert-based 
process Q 2 , special calculation procedure(s) Q 3 . 


• s 

= MkPkMkGkQ 





Si 

— II\ 'k 'k ]\/Ii 

*Gi 

k Qi, S2 = Ml k P2 

kr Ml kc Gi 

* Qi, 


S 3 

= M3 k Pi k M2 

*Gi 

k Qi, S4 = M3 k P3 

k Mq k G2 

* Qi, 


S 5 

— II2 'k kc M’j 

*G 3 

k Q2 
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Fig. 2.35. Illustration for morphological scheme of clustering 


Thus, five illustrative alternative examples of the composite (series-parallel) solving strategies for clus¬ 
tering are the following (Fig. 2.35): 

.S*! = Ml 'k F*i k Ml k Gi k Qi = (Ai k F2) k {Ui k V2 k IFi) k Mi k Gi k Qi‘, 

52 = Ml k P2 k Ml k Gi k Qi = {Xi k I2) * {U2 * Vi * IF2) * Ml * Gi k Qi', 

53 = H3 k Pi k M2 k Gi k Qi = (A7 * I2) * {Ul * V2 * Wi) k M2 k Gi k Qi', 

54 = M3 k P3 k Mg k G2 k Qi = (A7 k y^) k [JJ4 k Vi k IF2) k (M^2‘^Af3&Af4) k G2 k Ql] and 
S3 = M2 k P3 k M7 k G3 k Q2 = (A3 k (Fi&y^)) k (U4 kVi k W2) k (M2&M3&M5) k G3 k Qi. 

In Fig. 2.36, a graphical illustration for three composite strategies above is depicted. 


52 : 

54 ■■ 

55 : 


Ai ^ r2 

A 7 — ^2 


A3 


Ul V2 W2 Ml Gi 

^ M 2 

U4 ^ V2 ^ IF2 M3 G2 

M4^ 


U4 V2 


As 


M 2 ' 

W 2 ^ M 3 ■ 

Mg' 


G 3 


Fig. 2.36. Examples of composite solving strategies 


Qi 

Qi 

Qi 


2.4. On clustering in large scale data sets/networks 

In recent years, the significance of clustering in large-scale data bases and analysis and modeling in 
large networks has been increased, for example: 

(i) clustering of large data sets (e.g., |55I289I291I299I543I653I667] 1: 

(ii) detection of communities in large networks (e.g., |I27I230I285I286I370I495I65S] 1: 

(hi) detection of communities in mega-scale networks (e.g., [591600] 1: 

(iv) tracking evolving communities in large networks (e.g., m)- 

Table 2.22 illustrates some dimensional layers (classification) of data sets/networks. 
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Table 2.22. Dimensional layers of data sets/networks 


No. 

Type of studied data 
sets/networks 

Number of 
objects/ 
network nodes 

Examples of 
applications 

Some 

source(s) 

1. 

Simplified data sets/ 
networks 

(e.g., small groups) 

- 10...60 

(i) student group, 

(ii) sport club network, 

(hi) laboratory group, 

(iv) Web page structure, 

(v) product assortment 
(product variety) 

1646) 

2. 

Simple data sets/ 
networks 

- 100 

(i) university department, 

(ii) animal network, 

(hi) big firm department, 

(iv) department of 
government organization, 

(v) network of books/articles 
(close by topic (s)) 

(vi) social network of 
bottlenose dolphins, 

(vii) supply chain network, 

(viii) network of software 
system components, 

(ix) molecular structures, 

(x) manufacturing structures 

[461] 

3. 

Traditional data 
sets/networks 

- 1 k 

(i) citation networks, 

(ii) university network, 

(hi) collaboration network, 

(iv) urban systems, 

(v) consumers bases, 

(vi) multiple server 
computer systems 

[226] 

4. 

Large data sets/ 
networks 

- 10 k 

(i) research society network, 

(ii) sensor networks, 

(iii) manufacturing 
technology networks, 

(iv) Microarrays 

pi] 

5. 

Very large data 
sets/networks 

- 100 k 

(i) client bases, 

(ii) VLSI, 

(iii) medical patients bases. 

m 

6. 

Mega-scale data 
sets/networks 

- 1 M 

(i) university library, 

(ii) bases of editorial houses, 

[6W] 

7. 

Super-scale data 
sets/networks 

- 10 M 

(i) library networks, 

(ii) Internet-based shops, 

(iii) protein sequence databases 

m 

8. 

To-day’s/prospective 

Web-based 

data sets/networks 

- 100 M ...1 B 

(i) World Wide Web, 

(ii) social networks 
(e.g., Twitter, Facebook) 



2.5. Note on multidimensional scaling 

Many decades, multidimensional scaling approach is widely used in many domains (e.g., | 7 ()l 8 t)ll 43 l 
115015801645) 1. Here, an initial space of object parameters is transformed and simplified (by increasing 
its dimension, on the basis of optimization). As a result, obtained clusters are more “good”. Table 2.23 
contains some basic directions in multidimensional scaling researches. 
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Table 2.23. Directions in multidimensional scaling (methods, clustering, CS applications) 


No. 

Approaches, models 

Source(s) 

1. 

Basic methods in multidimensional scaling: 


1.1. 

Multidimensional scaling, general 

1701861143115()I24()I349I58()I 

1.2. 

Nonmetric multidimensional scaling 

128413501 

1.3. 

Least-squares multidimensional scaling 

1242] 

1.4. 

Application of convex analysis to multidimensional scaling 

1366| 

1.5. 

Global optimization in multidimensional scaling 

124112411 

1.6. 

Probabilistic multidimensional scaling 

|663| 

1.7. 

Genetic algorithms, evolutionary methods 
in multidimensional scaling 

m 

1.8. 

Multigrid multidimensional scaling 

m 

1.9. 

Gonfigural synthesis in multidimensional scaling 


1.10. 

Functional approach to data structure 
in multidimensional scaling 

[99] 

1.11. 

Gonvergence of methods in multidimensional scaling 

pHTj 

1.12. 

Distributed multidimensional scaling 

m 

2. 

Multidimensional clustering: 


2.1. 

Multidimensional clustering algorithms 

m 

2.2. 

Multidimensional scaling and data clustering: 


2.2.1. 

Multidimensional scaling and data clustering 

|2M| 

2.2.2. 

Multidimensional scaling: tree-fitting, and clustering 

(545| 

2.3. 

Multidimensional data clustering utilizing 
hybrid search strategies 


3. 

Gontemporary applications in CS: 


3.1. 

Multidimensional clustering in data mining 

m 

3.2. 

Graph drawing by multidimensional scaling 


3.3. 

Visualization 

l619l 

3.4. 

Multidimensional scaling in communication, sensor 
networks (node localization, location, positioning, etc.) 

111611 41 13621 
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3. Basic Structured Clustering Combinatorial Schemes 
3.1. Auxiliary problems 

Table 3.1 contains a list of main auxiliary problems for combinatorial clustering methods/procedures. 


Table 3.1. Auxiliary problems 


No. 

Problem 

Clustering 

model(s)/stage(s) 

Solving schemes 

Some 

source(s) 

1. 

Transformation of 
scales 

Data processing 

1. Calculation 

2. Expert judgment 


2. 

Calculation of 
proximity matrix 

Main clustering 
schemes 

1. Direct calculation 

2. Calculation with 
scale transformation 


3. 

Multicriteria ranking/ 

l.Data processing 

1. Utility function 

|7111981324138713921 


sorting 

2. Clustering 

2.Pareto approach 

3.Outranking technique 

4.Expert judgment, etc. 

1485151816661 

4. 

Minimum spanning 

Graph-based 

l.Kruskal’s algorithms 

11011391174121712181 


tree 

clustering 

2.Boruvka’s algorithms 

159916061493162616391 

5. 

Knapsack-like problems 

k-means 

1. Dynamic programing 

117412181295132514201 


(basic problem, multi¬ 
ple choice problem, 
multicriteria problems) 

clustering, 

restructuring 

2. Approximation 
(e.g., FPTAS) 

3. Heuristics 

15231 

6. 

Assignment problems 

k-means 

l.Fast algorithms 

I16I79I90I174I218I 


(basic problem, 
generalized problem, 
multicriteria problem) 

clustering 

2. Heuristics 

S.Enumerative methods 

|292I323I351I392I396| 

148214831494151915321 

7. 

Covering problems 

Set covering 
based clustering 

1. Enumerative methods 

2. Heuristics 

I9I446I525I 

8. 

9. 

Dominating sets 
problem 

Partitioning problems: 

Dominating set 
based clustering 

1. Enumerative methods 

2. Heuristics 

|101I263I400I489I643| 

9.1. 

Graph partitioning 

Partitioning 
based clustering 

1. Approximation 

2. Heuristics 

113411661189I328I554I 

|625| 

9.2. 

Graph 

Correlation 

l.Enumerative methods 

|4I33I39I40I110| 

10. 

partitioning 

Communities detection: 

clustering 

2. Heuristics 

il59l345l564l662| 

10.1. 

Detection of clique/ 

Clique based 

l.Enumerative methods 

I2l8l 11 521176117412181 


quasi-clique 

clustering 

2. Heuristics 

I236I309I340I426I 

10.2. 

Detection of network 

Communities 

l.Enumerative methods 

I7I226I371I457I458I 


communities 

detection 

2. Heuristics 

|459I460I461I478I496| 

13. 

13.1. 

13.2. 

13.3. 

Finding agreement/ 
median/consensus for: 
Partitions 

Rankings 

Trees 

Consensus 

clustering 

1. Approximation 

2. Heuristics 

I28I44I107I264I293I 

I443I447I560I579I 

|47I135I136I392I574| 

112120119216031 

14. 

Morphological clique 
problem 

Clustering of 
multi-type objects 

1. Heuristic 

2. Enumerative methods 

il52l372l374l385l392| 


Fig. 3.1 depicts main stages for processing of initial data: 

1. Collection of initial information, i.e., set of objects A = {Ai,..., ..., A^}, set of parameters 

C = {Cl,..., Ci, ..., Cm}, estimates of objects upon parameters = (x,,^, ..., ..., (t = 1, n). 

2 . Calculation of distance matrix Z = ||zti,t2ll ('-i = l,n,C2 = l,n); (usually: complexity estimate 
equals 0 {m x n^)). 

3. Transformation of distance matrix into a spanning graph (if needed): 
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= INti.i. 2 ll G(A,r); (usually: complexity estimate equals 0(n^)). 



Fig. 3.1. Preliminary data processing 


Procedures for transformation of scales (e.g., transformation of ordinal scale into ordinal scale, trans¬ 
formation of vector-like scale into ordinals scale, transformation of vector scale into multiset based scale) 
can be useful for processing of proximity matrix and for design of covering graph (to obtain a simple 
covering graph, e.g., by the use of thresholds for egde/arcs estimates/weights). On the other hand, trans¬ 
formation of vector-like estimates into ordinal estimates can be based on multicriteria ranking (sorting 
problem). 

Minimum spanning tree problem is an important part of many effective solving schemes for many com¬ 
binatorial optimization problems (e.g., |139I218) 1: (a) polynomial approximation of initial graph by tree, 
(b) effective solving a combinatorial problem over the obtained tree. Here, several well-known algorithms 
for design of minimum spanning tree can be used, for example: Borovka’s algorithm Prim’s algorithm, 
Kruskal’s algorithm |10l217l218ll39l493l639j . Complexity estimate of the algorithms is: O(plogn) (or 
less |639) 1 {p is the number of edges, n is the number of vertices). 

Other auxiliary combinatorial problems (Table 3.1) are more complicated (i.e., they belong to class of 
NP-hard problems). Only in some simple cases polynomial algorithms can be used: 

(i) polynomial algorithms for basic assignment problem (e.g., |218I351) 1: 

(ii) polynomial approximate solving schemes for basic knapsack problem and multiple choice problem 
(e.g., )2l8l29bl32bl42T]IH^ l: 

(iii) polynomial algorithms for some network partition problems (e.g., cores decomposition of networks 

03 ); 

(iv) polynomial approximate solving schemes for simple cases of partitioning problems, network com¬ 
munity detection problems, covering problems. 

Thus, it is necessary to use polynomial heuristics or enumerative methods for the above-mentioned 
auxiliary combinatorial problems (i.e., generalized assignment problem, clique problems, morphological 
clique problem, dominating set problem, covering problems, graph partitioning problems, finding agree¬ 
ment/median/consensus problems, multicriteria combinatorial problems). In the case of fast clustering 
schemes, fast heuristics (e.g., some analogues of greedy algorithms) have to be used for auxiliary combi¬ 
natorial problems. 

Some basic clustering models (e.g., hierarchical clustering, k-means clustering) are often used as auxil¬ 
iary problems of multi-stage clustering schemes (e.g., for preliminary definition of a cluster set or cluster 
centroids). 

The significance of balanced clustering problems (by cluster size) has been increased in many domains 
(e.g., communication systems). As a result, balanced partition of tree problem can be useful components 
of contemporary clustering schemes. Generally, the problem of fc-balanced partitioning a tree is NP-hard 
{k is the number of elements in each cluster of clustering solution) |195] . Here, it may be reasonable to 
use fc-balanced agglomerative algorithm over a tree (i.e., under restriction on cluster sizes in obtained 
solution). 

3.2. Hierarchical clustering 

Hierarchical clustering is widely used in many domains (e.g., [146123212751300130213551375146815051 
I630| l. The approach consists in agglomerative (i.e., hierarchical, “Bottom-Up”) scheme (e.g., [27513001 
I302l375| l: 

1. Calculate the proximity (distance) matrix between elements. 

2. Start with n clusters containing one element. 
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3. Find the most similar pair of clusters from the proximity matrix and merge them into a single 
cluster 

4. Update the proximity matrix (reduce its order by one, by replacing the individual clusters with the 
merged clusters) 

5. Repeat steps 3,4 until a single cluster is obtained (i.e., n — 1 times). 

3.2.1. Basic agglomerative algorithm 

The basic agglomerative algorithm [algorithm 1) is the following (e.g., |375l392) b 

Stage 1 . Calculate the matrix of element pair V(A(ii), ^(^2)), ^(*1) G A, A(i2) G A, ii ^ 12 

“distances” (a simple case, metric h)'- 


m 



Stage 2. Searching for the minimum element pair “distance” = miuj^ j 2 g{i,...,ra} inte¬ 

gration of the corresponding two elements into a resultant “integrated” element, extension of the corre¬ 
sponding cluster. 

Stage 3. If all elements are processed than GO TO Stage 5. 

Stage 4- Recalculate the matrix of pair “distances” Z (initial element set is decreased by 1 element) 
and Go To Stage 2. 

Stage 5. Stop. 


As result, a tree-like structure for the element pair integration process (Bottom-Up) is obtained (one 
element pair integration at each integration step). A basic simplified procedure for aggregation of items 
(aggregation as average values) is as follows [Ji^,i 2 = Ai^&zAi^)\ Vj = l,m ^ 

Complexity estimates for the above-mentioned version hierarchical clustering algorithm (by stages) is 
presented in Table 3.2. 


Table 3.2. Complexity estimates for stages of basic hierarchical clustering algorithm 


Stage 

Description 

Gomplexity estimate 
(running time) 

Stage 1 

Galculate the distance matrix Z 

O(n^) 

Stage 2 

Searching for the minimum element pair “distance”, 
integration of the corresponding element pair, 
extension of the corresponding cluster 

0 ['n?) 

Stage 3. 

Ghecking the condition for stopping 
(all elements are processed) 

0 [n) 

Stage 4 

Recalculate the “distance” matrix Z 

O(n^) 

Stage 5. 

Stopping 

0(1) 


Here there exists a computing cycle (stages 2,3,4) that can contain [n — 1) steps. Thus, the general 
complexity (running time) of this hierarchical clustering algorithm equals 0[n^). Generally, hierarchi¬ 
cal clustering methods have the following problems: (a) sensitivity to noise and outliers, (b) difficulty 
handling different sized clusters and convex shapes, and (c) breaking large clusters. 

3.2.2. Balancing by cluster size 

Now let us consider a modified version of hierarchical clustering with a special requirement to cluster size 
(as balancing of cluster sizes) to obtain about the same (close) cluster sizes. Let B = {i?i,..., B^, ..., i?„} 
be the obtained set of clusters. Let at = |i3t| [i = 1 ,/v) be the size (i.e., number of elements) for cluster 
Bt. 

Thus for each cluster the following constraints are considered: o' < < a". For example: a' = 3, 

a" = 4. Evidently, one cluster of the obtained cluster set can contain less elements (i.e., 1,2,.., [a' — 1)). 
Generally, the above-mentioned requirement leads to balanced clustering solution by cluster size. This is 
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significant in many applications (e.g., local areas in communication networks, student teams in educational 
process). Our modified balanced by cluster size hierarchical clustering algorithm is: 

Stage 1. Calculate the matrix of element pair V(A(ii), ^( 12 )), A{ii) e A, A{i 2 ) G A, ii ^ 12 
“distances” (a simple case, metric ^ 2 ): 

Stage 2. Searching for the minimum element pair “distance” = miuj^ j 2 e{i,...,n} inte¬ 

gration of the corresponding two elements into a resultant “integrated” element, extension of the corre¬ 
sponding cluster. 

Stage 3. Analysis of the obtained extended cluster by new size a^. If = a" then deleting the 
cluster and its elements for the future processing (as a part of the resultant solution). 

Stage 4- If all elements are processed than GO TO Stage 6. 

Stage 5. Recalculate the matrix of pair “distances” Z (initial element set is decreased by I element) 
and Go To Stage 2. 

Stage 6. The other elements are organized as additional separated clusters (if needed). Stop. 

Complexity estimates for the above-mentioned version hierarchical clustering algorithm (by stages) is 
presented in Table 3.3. 

Table 3.3. Complexity estimates of stages for balanced (by cluster size) hierarchical algorithm 


Stage 

Description 

Complexity estimate 
(running time) 

Stage 1 

Calculate distance matrix Z 

0(}?) 

Stage 2 

Searching for the minimum element pair “distance”, 
integration of the corresponding element pair, 
extension of the corresponding cluster 

o\ri^) 

Stage 3. 

Checking the condition for stopping 
(all elements are processed) 

0{n) 

Stage 4. 

Analysis of the obtained cluster by cluster size 

0(1) 

Stage 5 

Recalculate the “distance” matrix Z 

O(n^) 

Stage 6. 

Stopping 

0(1) 


Here there exists a computing cycle (stages 2,3,4,5) that can contain (n — 1) steps. Thus, the general 
complexity (running time) of this hierarchical clustering algorithm equals O(n^). Note the average 
complexity estimate is less. 

Example 3.1. A numerical example of tree with weights of edges (i.e., proximity between element 
pairs) is presented in Fig. 3.2: tree T = (A, A), A = {1,2,3,4,5,6,7,8,9,10,11,12,13,14}. The con¬ 
straint for cluster size is: < 3. Table 3.4. contains the corresponding proximity matrix, i.e., weights of 
edges (a very large proximity is denoted by symbol proximity is symmetric). 

Cluster ^^^^^Cluster Jfs 
Cluster ) ^ 

/ Cluster Cluster 

la'^^ld ATs Ag 

Cluster X\ 

Fig. 3.2. Balanced clustering of tree 
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Table 3.4. Proximities for tree-like example (edge ( 11 , 12 )) 


ii 

h ■ 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

1 


1.5 

1.7 

0.5 

0.2 

k 

k 


k 

■*r 

■*: 

k 


k 

2 



'k 

k 

k 

0.1 

0.6 

k 

k 

k 

k 

k 


k 

3 




k 

k 

k 

k 

3.1 

1.0 

0.9 

k 


k 

k 

4 





k 

k 

k 


■*: 

■*r 

k 

k 

k 

k 

5 






k 

k 

k 

■*: 


2.5 

2.1 

k 

k 

6 







k 

k 

k 


k 

k 

k 

k 

7 








k 

k 

k 

k 

k 

k 

k 

8 









k 

■*r 

k 

k 

0.3 

0.4 

9 










k 

k 

k 

k 


10 











k 

k 

k 


11 












k 

k 

k 

12 













k 

k 

13 














k 


In the considered case (i.e., tree), proximity information can be presented as a list of (n—1) components: 
number of vertex, number of “son”-vertex, weight of the corresponding edge (Table 3.5). 


Table 3.5. Initial list if proximities 


Vertex ii 

Vertex 12 
(“son” ofii) 

Weight of 
edge ( 11 , 12 ) 

1 

2 

1.5 

1 

3 

1.7 

1 

4 

0.5 

1 

5 

0.2 

2 

6 

0.1 

2 

7 

0.6 

3 

8 

3.1 

3 

9 

1.0 

3 

10 

0.9 

5 

11 

2.5 

5 

12 

2.1 

8 

13 

0.3 

8 

14 

0.4 


In the example, proximity between element x and cluster Y (or integrated element) T) is used 

(case 2 from previous section 2). The steps of agglomerative algorithms to obtain the balanced clustering 
(cluster size is < 3) are the following: 

Step 1. Integration of elements 2 and 6 into J(2,6). As a result. Table 3.6 is obtained. 


Table 3.6. List of proximities after step 1 


Vertex ii 

Vertex 12 
(“son” ofzi) 

Weight of 
edge (11,12) 

1 

J(2,6) 

1.5 

1 

3 

1.7 

1 

4 

0.5 

1 

5 

0.2 

J(2,6) 

7 

0.6 

3 

8 

3.1 

3 

9 

1.0 

3 

10 

0.9 

5 

11 

2.5 

5 

12 

2.1 

8 

13 

0.3 

8 

14 

0.4 
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Step 2. Integration of elements 1 and 5 into J(l,5). As a result, Table 3.7 is obtained. 


Table 3.7. List of proximities after step 2 


Vertex ii 

Vertex 12 
(“son” ofti) 

Weight of 
edge (ii,i 2 ) 

J(l,5) 

J(2,6) 

1.5 

J(l,5) 

3 

1.7 

J(l,5) 

4 

0.5 

J(2,6) 

7 

0.6 

3 

8 

3.1 

3 

9 

1.0 

3 

10 

0.9 

J(l,5) 

11 

2.5 

J(l,5) 

12 

2.1 

8 

13 

0.3 

8 

14 

0.4 


Step 3. Integration of elements 8 and 13 into J(8,13). As a result, Table 3.8 is obtained. 


Table 3.8. List of proximities after step 3 


Vertex ii 

Vertex 12 
(“son” ofti) 

Weight of 
edge ( 11 , 12 ) 

J(l,5) 

J(2,6) 

1.5 

J(l,5) 

3 

1.7 

J(l,5) 

4 

0.5 

J(2,6) 

7 

0.6 

3 

J(8,13) 

3.1 

3 

9 

1.0 

3 

10 

0.9 

J(l,5) 

11 

2.5 

J(l,5) 

12 

2.1 

J(8,13) 

14 

0.4 


Step 4-. Integration of elements J(8,13) and 14 into J(8,13,14). Thus, cluster 1 is designed Xi = 
{8,13,14}. The corresponding elements (i.e., J(8,13),14) can be deleted from the next analysis. As a 
result. Table 3.9 is obtained. 


Table 3.9. List of proximities after step 4 


Vertex ii 

Vertex 12 
(“son” ofti) 

Weight of 
edge ( 11 , 12 ) 

J(l,5) 

J(2,6) 

1.5 

J(l,5) 

3 

1.7 

J(l,5) 

4 

0.5 

J(2,6) 

7 

0.6 

3 

9 

1.0 

3 

10 

0.9 

J(l,5) 

11 

2.5 

J(l,5) 

12 

2.1 


Step 5. Integration of elements J(l,5) and 4 into J(l,4, 5). Thus, cluster 2 is designed X 2 = {1,4,5}. 
The corresponding elements (i.e., J(l,5),4) can be deleted from the next analysis. As a result, Table 
3.10 is obtained. 
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Table 3.10. List of proximities after step 5 


Vertex ii 

Vertex 12 

Weight of 


(“son” ofzi) 

edge (^ 1 , 12 ) 

J(2,6) 

7 

0.6 

3 

9 

1.0 

3 

10 

0.9 


Step 6 . Integration of elements J{2,6) and 7 into J(2, 6 , 7). Thus, cluster 3 is designed X 3 = {2, 6 , 7}. 
The corresponding elements (i.e., J(2,6),7) can be deleted from the next analysis. As a result, Table 
3.11 is obtained. 


Table 3.11. List of proximities after step 6 


Vertex ii 

Vertex 12 

Weight of 


(“son” ofii) 

edge {ii.h) 

3 

9 

1.0 

3 

10 

0.9 


Step 7. Integration of elements 3 and 10 into J(3,10). As a result. Table 3.12 is obtained. 


Table 3.12. List of proximities after step 7 


Vertex ii 

Vertex 12 
(“son” ofii) 

Weight of 
edge { 11 , 12 ) 

J(3,10) 

9 

1.0 


Step 8 . Integration of elements J(3,10 and 9 into J(3, 9,10). Thus, cluster 4 is designed X 4 = {3,9,10}. 
The corresponding elements (i.e., J(3,10), 9) can be deleted from the next analysis. 

Two separated elements 11 and 12 can be organized as two clusters: A 5 = {11} and Xq = {12}. 

3.2.3. Improvements of hierarchical clustering scheme |375j 

First, let us point out some properties of the considered clustering process as follows: 

1. The matrix of element pair proximity can contain several “minimal” elements. Thus there are 
problems as follows: (i) selection of the best unit pair for integration; (ii) possible integration of several 
unit pair at each algorithm stage. 

2. Computing the matrix of element pair distances often does not correspond to the problem context 
and it is reasonable to consider a “softer” approach for computing element pair proximity. 

3. The obtained structure of the clustering process is a tree. Often the clustering problem is used to 
get a system structure that corresponds to the above-mentioned clustering process [e.g., evolution trees, 
system architecture). Thus, it is often reasonable to organize the clustering process as a hierarchy, e.g., for 
modular systems in which the same modules can be integrated into different system components/parts. 

Fig. 3.3 illustrates concurrent integration of element pairs at the same step of the algorithm when some 
elements can be integrated into different system components parts, i.e., obtaining a hierarchical system 
structure (common modules/parts, e.g., 3 and 4). In this case, obtained clusters can have intersections 
(Fig. 3.4). 


C 1,2,3,4,5,6,7, 8 ) 




t \ 


Step 2 


(l,3,4,6,7,8)r4X^ 

7,8) 

Step 1 

nn r2^ 


3,7,8) 

Step 0 

1 2 

3 4 5 

3 7 8 


Fig. 3.3. 

Illustration for hierarchy 



Fig. 3.4. Clusters 
for Step 2 (Fig. 3.2) 
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Now, let us describe some possible algorithm improvements. 

Improvement 1 (algorithm 2); 

Stage 1. Computing an ordinal distance/proximity (0 corresponds to equal or the more similar ele¬ 
ments). Here it is possible to compute the pair distance/proximity via the previous approach and mapping 
the pair distance to the ordinal scale. 

Stage 2. Revelation of the smallest pair distance and integration of the corresponding elements. 

Note 1. It is possible to reveal several close element pairs and execution several pair integration. 

Note 2. It is possible to include the same element into different integrated pairs. 

Stage 3. The stage corresponds to stage 3 in algorithm 1. 

Here, a hierarchical structure for the element pair integration (Bottom-Up) is obtained (several element 
pair integration at each integration step). The complexity of the problem may consist in revelation of 
many subcliques (in graph over elements and their proximity). In the process of computing the ordinal 
proximity it is reasonable to use a limited number of element pairs for each level of the proximity ordinal 
scale. As a result, the limited number of integrated element pairs (or complete subgraphs or cliques) will 
be revealed at each integration stage. This provides polynomial complexity of the algorithm (number of 
operations, volume of required memory) 0 {m tt3). 

Improvement 2 (algorithm 3): This algorithm is close to algorithm 2, but the computing process for 
the ordinal element pairs proximity is based on multicriteria analysis, e.g., Pareto-approach or outranking 
technique (i.e., Electre-like methods). Complexity of the algorithm is 0{m n^). 

The algorithms 2 and 3 implement the following trend: 

from tree-like structure (of clustering process) to hierarchy. 

An analysis of obtained clique(s) can be included into the algorithms as well. 

3.3. K-means clustering 

K-means clustering approach is widely used |161I276I300I302I299I318I435] . The basic simplified version 
of the algorithm is: 

Stage 1. Select K points as initial centroids (e.g., mean points) (e.g., selection is based on random 
process). 

Stage 2. Cycle by all n points: 

(2.1) Form K clusters by assigning all points to the closest centroid. 

(2.2) Recalculate the centroid of each cluster. 

(2.3) If all points are assigned GO TO stage 3. 

(2.4) GO TO (2.1). 

Stage 3. Stop. 

Complexity estimates for AT-mean clustering algorithm (by stages) is presented in Table 3.13. 


Table 3.13. Complexity estimates of stages of AT-means clustering algorithm 


Stage 

Description 

Complexity estimate 
(running time) 

Stage 1 

Selection of AT centroids 

0{K) 

Stage 2.2 

Assignment of all n points to AT centroids 
(by m attributes) 

0{n X K X to) 

Stage 2.2 

Recalculation of the centroids (for K clusters) 

0{K X n X to) 

Stage 2.3. 

Checking the condition for stopping 
(all elements are processed) 

0(1) 

Stage 3. 

Stopping 

0(1) 


Thus, the general complexity of this algorithm equals 0{K x n x m). 
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This approach has some problems when initial object set contains “outlier”-like point(s). (Fig. 2.14). 
A general framework of k-means clustering is shown in Fig. 3.5. 


Feedback 



Fig. 3.5. General framework of k-means clustering process 


3.4. Clustering as assignment 

Generally, the k-means clustering approaches involve a stage to assignment the items to preliminary 
defined clusters. Thus assignment problems can be used at this stage and, as a result, special assignment 
based clustering methods are obtained (e.g., |227l292) b 

In basic assignment problem (bipartite matching problem) there are the following: items/elements 
{1,..., i,..., n}, agents {1,..., j,...,/x}, positive profit for assignment of item i to agent j , binary 
variable Xij equals 1 if item i is assigned to agent j and 0 otherwise. The basic assignment problem is 
(e.g., [ZE]): 

^ n n l-i 

max EE CtjXij s.t. '^aijXij <1, j = 1,/x, '^Xij <1, i = l,n, Xij G {0,1}, i = l,n, j = 1,/x. 

j—1i—1 i—1 j—^ 

Here each item has to be assigned to the only one cluster (agent). There exist several well-known 
polynomial algorithms for the problem above (e.g., |183ll39ll67l218l3M] b 

The generalized assignment problem GAP can be described as a multiple knapsack (or multiple agents) 
problem. Analogically, given n items/elements (i = l,n) and /x knapsacks (agents) {j = 1,/x). The 
following notations are used: Cy is a profit of item i if it is assigned to knapsack (agent), Uij is a weight 
(e.g., required resource) of item i if it assigned to knapsack (agent) j, bj is a capacity (volume of resource) 
of knapsack (agent) j, binary variable Xij equals 1 if item i is assigned to agent j and 0 otherwise. Clearly, 
the knapsack (agent) capacity can be considered as multiple recourse (i.e, a vector-like parameter) as well 
(e.g., [219) 1. The problem is [2181420] : 

Assign each item to exactly one knapsack so as to maximize the total profit assigned, without assigning 
to any knapsack a total weight greater than its capacity. 

The basic problem statement is: 

II n n fi 

max ^ ^ ^ ^ijXij s.t. ^ ( ^ijXjj f^bj, j l,ix, ^ ^ Xjj ^ 1, x ^,n, x^j G {0,1}, z — j — \,p,. 

j—1 i—1 i—1 j—1 

The problem is known to be NP-hard (e.g., [524) 1. Evidently, the objective function can be minimized 
as well (e.g., minumum cost assignment of a set of items/objects to a set of agents). In the case of 
ttij = 1 \/i,\/j, each agent has an integer restristion of assigned elements as restriction for cluster size). In 
multiple assignment problem constraint J2j=i^ij < 1 (* = 1, "ra) is replaced by J2j=i^ij ^ ^i (* = 
where Xi is restriction for the number of admissible assignment to different agents for elements i. In 
applications, knapsack/agents can be considered as service centers (e.g., access points in communication 
networks) which have limited service resource(s) (e.g., [3961397) 1. 

Generally, the following kinds of algorithms have been used for basic generalized assignment problems 
(e.g., [8713251420) 1: (i) exact algorithms as enumerative methods (e.g.. branch-and-bound algorithms) 
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(e.g., [50315151531] ^. (ii) relaxation methods, reduction algorithms (e.g., relaxations to linear programming 
models, relaxation heuristics) (e.g., [219l408l581j l. (iii) approximation schemes (e.g., [130I151I200I4661 
1547] 1. (iv) various heuristics (e.g., [2531455] 1 including greedy algorithms (e.g., [4551510] !. set partitioning 
heuristic (e.g., [55]) genetic algorithms (e.g., [123] !. tabu search algorithms (e.g., [1621281] !. Fig. 3.6 
illustrates the generalized assignment problem. 


Restriction 


Items 



for agent 1 





Agent 1 

(knapsack, 

cluster) 


Agent j 

(knapsack, 

cluster) 


Agent /r 

(knapsack, 

cluster) 


Fig. 3.6. Illustration of generalized assignment 


In multiple criteria generalized assignment problem, vector-like profit is considered for each item i (e.g., 
|396I392|532|655| !: c,, = (cb,..., ..., 4) (criteria: {Ci,..., Q,..., Cfe}). 

A simplified multicriteria problem statement can be examined as follows: 

^ n fi n fi n 

max EE cljXij, ... ,max EE c\jXij, ... ,max 

j—l i—l 3 — ^ i—1 i—1 2—1 

n [J. 

s.t. ^ ^ QijXjj j — 1,/i, ^ ^ Xjj ^1, i — 1, n, Xij G {0,1}, ! = l,7r, j — 1,/r. 

i=l j=l 

Here it is reasonable to search for Pareto-efficient solutions. 


Example 3.2. A modified numerical example for connection of end-users and access points in a 
wireless telecommunication network is based on the example from [39613971392] . Four access points are 
considered as cluster centroids and 14 end-users are examined as initial objects (example from [396] is 
compressed. Fig. 3.7). 




12 



7 

(§ 

13 


3 





(i) 

9 


@14 



@ 

6 

@ 

11 


Fig. 3.7. Example: 4 access points, 14 users 


Let {1, ...,i, ...,n} be a set of users (here: 14) and {1, ...,j, ...,/r} be a set of access points (here: 4). 
Each user i is described by parameters (a compressed set of parameters): (i) coordinates {xi,yi,Xi); 
(ii) parameter corresponding to required frequency bandwidth (e.g., 1 Mbit/s ... 10 Mbit/s) fi] (iii) 
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maximal possible number of access points for connection Ki < fx (here Ki = 1 \/i, i.e., each user is assigned 
to the only one access point/cluster); (iv) required reliability of information transmission r*. Note, in 
multi-assignment problem k* > 1. 

Each access point is described by parameters (a compressed set of parameters): (a) coordinates of access 
point {xj,yj,Xj), (b) parameter corresponding to maximal possible traffic (i.e., maximum of possible 
bandwidth) fj , (c) maximal possible number of users under service kj , (d) reliability of channel for data 
transmission Vj, (e) admissible proximity (distance) dj. Table 3.14 and Table 3.15 contains parameters 
estimates of the access points and users. 


Table 3.14. Parameter estimates of access points 


Access 

point 

j 

Coordinates: Xj yj Zj 

Bandwidth 

fj 

Number 
of users 

Uj 

Reliability 

C 

Admissible 

distance 

dj 

1 

50 157 10 

30 

4 

10 

10 

2 

150 165 10 

30 

5 

15 

10 

3 

72 102 10 

42 

6 

10 

6 

4 

140 112 10 

32 

5 

8 

9 


Table 3.15. Parameter estimates of end users 


User 

i 

Coordinates: Xj 

Vj 


Bandwidth 

h 

Reliability 

Ti 

1 

30 

165 

5 

10 

5 

2 

58 

174 

5 

5 

9 

3 

88 

156 

0 

6 

6 

4 

no 

169 

5 

7 

5 

5 

145 

181 

3 

5 

4 

6 

170 

161 

5 

7 

4 

7 

52 

134 

5 

6 

8 

8 

86 

134 

3 

6 

7 

9 

120 

140 

6 

4 

6 

10 

150 

136 

3 

6 

7 

11 

175 

125 

1 

8 

5 

12 

27 

109 

7 

8 

5 

13 

55 

105 

2 

7 

10 

14 

98 

89 

3 

10 

10 


As a result, each pair “user-access point” can be described by the following parameters (a compressed 
set of parameters): (1) proximity (e.g., Euclidean distance) (2) level of reliability r^, (3) parameter 
of using bandwidth fij. 

Clearly, Euclidean distances between users and access points {dij} can be calculated on the basis 
coordinates from Table 3.14 and Table 3.15. Thus, the following parameter vector is obtained = 
{dij,rij, fij) (i = l,n,j = 1,/r). Further, the parameter vector can be transformed into a profit Cy 
(i.e., mapping of vector estimate into ordinal scale [1,2,3], 3 corresponds to the best level, multicriteria 
ranking based on outranking ELECTRE technique can be used). 

The assignment of user i to access point j is defined by Boolean variable Xij (xij = 1 in the case 
of assignment i to j and Xij = 0 otherwise). The assignment solution is defined by Boolean matrix 
X = llxijll, i = l,n, j = 1,/i. Finally, the problem is: 

n 

max E E Cij Xij 

1=1 i=l 

n n f-L 

S.t. ^ ^ — fj ^ ^ ^ij E kj Vj = ^ ^ ^ij E 1 Vz = 1,715 

i—1 i—1 j—^ 

Xij =0 if dij > dj V 7 = 1, n, V j = 1, /i; Xij = 0 if Vij > Vj V z = 1, n, V j = 1, 
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Xij G {0,1} y i = 1, n, y j = 1, fi. 

A numerical example of the assignment solution (i.e., clustering solution) is depicted in Fig. 3.8. In 
[396] , the described problem is examined as generalized multiple assignment problem (extended version). 



Fig. 3.8. Assignment of users to access points 

3.5. Graph based clustering 

3.5.1. Minimum spanning tree based clustering 

The preliminary building of minimum trees is widely used in many combinatorial problems (e.g., |218j l. 
The algorithmic complexity estimate for this spanning problem over graph equals 0(n log n) (n is the 
number of graph vertices). Minimum spanning tree based clustering algorithms have been studied and 
applied by many researchers (e.g., |235l244l353l445l479l492l555lti0til(i2fil(iBn] l. The basic stages of the 
algorithms are as follows: 

Stage 1. Calculation of distance/proximity matrix Z. 

Stage 2. Design of the corresponding graph G. 

Stage 3. Building of the minimum spanning tree T for graph G. 

Stage 4- Clustering of the vertices of tree T (e.g., by algorithm of deletion of branches, by algorithm 
of hierarchical clustering). 

Stage 5. Stopping. 

Further, the usage of hierarchical clustering at stage 4 is considered. Complexity estimates for minimum 
spanning tree clustering algorithm (by stages) are presented in Table 3.16. 

Table 3.16. Complexity estimates of stages for minimum spanning tree based clustering 


Stage 

Description 

Complexity estimate 
(running time) 

Stage 1. 
Stage 2. 
Stage 3. 
Stage 4. 
Stage 5. 

Calculate distance matrix Z 

Design the corresponding graph G 
Building the minimum spanning tree 
Clustering of the tree vertices 
Stopping 

O(n^) 

0 (n^) 

0 (n logn) 

0 (n logn) 

0 ( 1 ) 


Stages 3, 4, 5 correspond to the situation when a graph is examined as initial data. In this case, 
complexity of the algorithm equals 0 (n log n). 

Example 3.3. A numerical example for elements A = {1,2,3,4,5,6,7,8,9,10,11,12} illustrates 
building of graph G = {A,E) (corresponding proximity matrix Z), minimum spanning tree T = (A,E'), 
and clustering solution X = {Xi,X 2 ,X 3 } (cluster size < 4). The following is implemented: 

Proximity matrix Z => Graph G = {A, E) Tree T = (A, E') Clustering solution X = (Ai, X 2 , A 3 }. 

The method for building the spanning tree is used as in example 3.1. Table 3.17 contains proximity 
matrix (symbol corresponds to a very big value). 
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Table 3.17. Proximities for example (edge { 11 , 12 )) 


ii 

*2 : 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

1 


0.3 

1.4 

1.45 

k 


k 

k 

k 


k 


2 



0.3 

'k 


2.6 

0.2 

1.8 

■*: 


k 


3 




0.4 


k 

1.65 

0.25 

k 


k 


4 





0.4 

k 

k 

0.45 

1.9 

k 

k 


5 






k 

k 

■*: 

0.35 

1.5 

k 

k 

6 







0.1 

k 

k 


1.4 


7 








0.41 

k 


0.4 


8 









0.9 

k 

2.1 


9 










0.15 

k 

0.5 

10 











k 

2.0 

11 












2.5 


Fig. 3.9 depicts corresponding graph, G = {A,E), Fig. 3.10 depicts spanning tree T = {A,E'), 
and clustering solution. Generally, it is reasonable to point out threshold based modification of graph 
G = {A, E) over object set A: deletion of edges by condition: the weight “>” the threshold. Decreasing 
the threshold leads to decreasing the cardinality of E. This process can be very useful for analysis and 
processing of initial data in clustering. Let us consider an illustration of the above-mentioned process 
on the basis graph from example 3.3 (basic proximity matrix from Table 3.17): (i) threshold equals 
2.6: graph G = {A, E) in Fig. 3.9; (ii) threshold equals 1.4: graph G^ = {A,E^) in Fig. 3.11; (iii) 
threshold equals 0.5: graph G^ = (A, E'^) = T = {A, E') in Fig. 3.10 (here: spanning tree); (iv) threshold 
equals 0.3: graph G^ = {A,E^) in Fig. 3.12. 






11 * *12 
Fig. 3.12. Graph G^ = {A,E^) 


As a result, a useful structure can be found. The described by example procedure is an important 
auxiliary problem. An additional significant problem consists in analysis of the obtained graph, for 
example: (a) connectivity, (b) similarity to tree (or hierarchy, clique). Further, a modified version of 
adaptive minimum spanning tree clustering algorithm is examined as follows. Initial data are: (a) set of 
objects/alternatives A = {Ai,..., Aj,..., A„}, (b) set of parameters/criteria C = {Gi, ...,Gm}, (c) 

estimate matrix X = {xij}, i = l,n, j = 1, m, where Xij is estimate of Ai upon criterion Cj (a qualitative 
scale is considered). The algorithm consists of the following stages: 
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Stage 1. Calculation of proximity matrix Z = {zik\, i = l,n, k = l,n, where Zik is estimate of 
proximity (distance) between Ai and Ak (e.g., Euclidean metric is used). Evidently, zu = 0,V* = l,n. 

Stage 2. Transformation of matrix Z into ordinal matrix Y = {yik}- Let us consider the maximum and 
minimum values of elements of matrix Z: 2 ;”^“ = {zik}, 2 ™“=^ = maxy^^j-yi {zik}- 

Thus an interval is obtained [z™*", 2 ’”“®] and d = — 2:™“. Now an additional integer parameter S 

(e.g., 3,4, 5,6) is used. Let <5 = 5. Then elements of new matrix Y (i.e, adjacency matrix) are based on 
the following calculation: 


yik = 


0 , 

1 , 

2 , 

3 , 

4 , 


if 0.0 < Zik < d/s, 

if d/s < Zik < 2d/S, 
if 2d/S < Zik < 3d/S, 
if 3d/S < Zik < 4/d, 
if Sd/S < Zik < d. 


Stage 3. Obtaining an interconnected graph over elements A (iterative approach): 
Let A = 1, 2,... be an integer algorithmic parameter (for the algorithm cycle). 
Step 3.1. Initial value A = 1. 

Step 3.2 Transformation of ordinal matrix Y into Boolean matrix B = {bik}'. 


L. _ / 1, if y^k < A, 

\ 0, if yik > A. 

Step 3.3. Building a graph over elements A: = {A, T^), where T^ is the set of edges, edge {Ai,Ak) 

exists if bik = 1. 

Step 3 . 4 . Analysis of connectivity for graph = (A,r^). If the graph is connected, then GOTO 
Step 3.6. 

Step 3.5. A = A + 1 and GOTO Step 3.2 

Step 3.6. Building of minimum spanning tree for graph G‘^ = (A, T^): T‘^ = {A,E^). 

Here, several well-known algorithms can be used, for example: Borovka’s algorithm Prim’s algorithm, 
Kruskal’s algorithm |10I217I218I139I493I639] . Complexity estimate of the algorithms is: 0{p log n) (or 
less |639) 1 (p is the number of edges, n is the number of vertices). It is necessary to take into account for 
each edge 7 S T its weight as follows: proximity value Zik for corresponding element of Z. 

Step 3.7. Clustering set A on the basis of spanning tree = {A, E^) while taking into account an 
algorithmic parameter: a number of elements a in each obtained cluster a' < a < a", for example a' = 4, 
a" — 6 . The constrains above have to be based on the engineering analysis of the applied problem. 

Stage 4- Stop. 


Complexity estimates for the described adaptive algorithm (by stages) is presented in Table 3.18. Thus, 
the general complexity estimate (running time) of the described adaptive algorithm equals O(n^). 

Generally, the problem of fc-balanced partitioning a tree is NP-hard {k is the number of elements in 
each cluster of clustering solution) |195) . 

Note, the obtained clustering solution has a property: “modularity”. This can be very important for 
many applied problems (e.g., close cardinalities of clusters/groups: local region elements in communica¬ 
tion network, student teams). 
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Table 3.18. Complexity estimates for adaptive minimum spanning tree based algorithm 


Stage/step 

Description 

Complexity estimate 
(running time) 

Stage 1 

Calculate the distance matrix Z 

0 {n"‘) 

Stage 2 

Transformation of matrix Z into ordinal matrix Y 

O(n^) 

Stage 3 

Design of interconnected graph over elements A 

0 {'n?) 

Step 3.1 

Specifying the start of the cycle 

0(1) 

Step 3.2 

Transformation of matrix Y into Boolean matrix B 

0 {v?) 

Step 3.3 

Building the graph G that corresponds to matrix B 

O(n^) 

Step 3.4 

Analysis of connectivity of graph G 

0 {n) 

Step 3.5 

Correction of cycle parameter 

0(1) 

Step 3.6 

Building the minimum spanning tree T for graph G 

O(plogn) 

Step 3.7 

Clustering the vertices (elements A) of the spanning tree T 
while taking into account the constraints for cluster size 

0 {n) 

Stage 4. 

Stopping 

0(1) 


3.5.2. Clique based clustering 

Here, an initial graph G = {A, E) is examined as initial data. In a clique (complete graph/subgraph), 
each vertex is connected to all other the vertices (Fig. 3.13). A quasi-clique can be examined, for 
example, as a clique without one-two edges. The cliques (or quasi-cliques) form a very strong clusters 
(from the viewpoint of interconnection). The problem of finding a maximal clique in a graph is a well- 
known NP-hard problem (e.g., |218l319] b Thus, heuristics or enumerative methods have been used for 
the problem. 



One-vertex Two-vertex Three-vertex Four-vertex Five-vertex 
clique clique clique clique clique 

Fig. 3.13. Illustration of cliques 


Clique-based clustering process can be organized as a series of clique problems |218) : 

Stage 1. Finding the “maximal clique” (or maximal “quasi-clique”) in graph G = {A,E): subgraph 
H = {B,V) (H CA,V C E). 

Stage 2. Forming a cluster from subgraph iJ and compression of initial graph G: G' = {A',E'), 
{A' = A\H, E' = E\{V[JW}^ where IF is a set of external edges of clique, i.e., the only one vertex 
belongs to set E[) (Fig. 3.14). 

Stage 3. If G' is empty GO TO Stage 4 otherwise GO TO Stage 1. 

Stage 4- Stop. 


External edges IF 

/ \ 


Clique 
H = iB,V) 


Graph 

G = (a,E)J 


Fig. 3.14. Illustration of clique in graph 


The above-mentioned solving scheme is based on series of NP-hard problems. Evidently, it is possible 
to find several “maximal cliques” concurrently. Some sources on researches on clique finding and clique 
based clustering are presented in Table 3.19. 

Clique partitioning problem for a given graph G = {A, E) with edge weights consists in partitioning 
the graph into cliques such that the sum of the edge weights over all cliques formed is as large as possible 
(e.g., I34OI4690 . 
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There are some close problems over graphs/digraphs, for example, independent set problems and dom¬ 
inating set problems which are used in clustering as well (e.g., |105ll31ll32ll40l263l296l4g5] b Recently, 
the significance of dynamic problems over data streams has been increased including clique/quasi-clique 
finding in graph streams (e.g. |8I128I247I389] 1. 

On the other hand, clique-based approaches can be considered as density-based and grid-based clus¬ 
tering methods. In some recent works, subgraph as clique/qiasi-clique is considered as one of network 
community structures (network community based clustering |226I459I460I496 ] ). 


Table 3.19. Detection of cliques/quasi-cliques and clustering 


No. 

Research 

Source(s) 

1. 

Detection and analysis of cliques in graphs: 


1.1 

Cliques in graphs 

|309I444| 

1.2. 

Finding of clique/quasi-clique in graphs 

1211818111931218147214841 

1.3. 

Maximum-weight clique problem 

j29l34l«8l473l48l| 

1.4. 

Finding all cliques of undirected graph 

m 

1.5. 

Enumeration maximal cliques of large graph 

m 

2. 

Clustering based on cliques: 


2.1. 

Clique based clustering 

|9l55l17t)l23fil3()2l42fil437l539l588| 

2.2. 

Clique partitioning problem 
(clique partition of maximum weight) 

ll 54134014091 

3. 

Clique-based multiple clustering: 


3.1. 

Ensemble clustering with voting active clusters 

[Wi] 

3.2. 

Cliques for combining multiple clusterings 

m 

4. 

Clique based methods over data streams: 


4.1. 

K-clique clustering in dynamic networks 

m 

4.2. 

Clique-based fusion of graph streams 

[389| 


In recent decades, several new combinatorial problems as clique clustering in multipartite graphs have 
been suggested (e.g., |96I152I272I372I374I392I596] 1. Fig. 3.15. illustrates this kind of problems. Table 
3.20 contains a list of the research directions in the above-mentioned field. 



Fig. 3.15. Cliques in four-partite graph 


Table 3.20. Research directions in multi-partite graphs 


No. 

Research 

Source(s) 

1. 

Problem of compatible representatives 

|339| 

2. 

Morphological clique (ordinal estimates) 

|372I374I392| 

3. 

Morphological clique (multiset estimates) 

138013921 

4. 

Clustering in multipartite graph 

19015901 

5. 

Bipartite and multipartite clique problems 

m 

6. 

Morphological clique over graph streams 

|389| 

7. 

Coreset problems 

|194I272| 

8. 

Coresets in dynamic data streams 

|2()8| 
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3.5.3. Correlation clustering 

Correlation clustering provides a method for partitioning a fully connected labeled graph (label “+” 
corresponds to edge for similar vertices, label corresponds to edge for different vertices) while taking 
into account two objectives for the obtained clusters: 

(i) minimizing disagreements (i.e., minimizing the number of edges within the clusters > 

min) or maximizing the number of between clusters), 

(ii) maximizing agreements (i.e., the number of “+” edges insides the clusters) max) (e.g., 

|4l33l4()l52ll59l345l5f)4lh62) i. 

In the basic above-mentioned problem formulation, the objective functions are summarized. In other 
words, binary scale [—1,-|-1] is used for each edge as a weight (zero value is not used). Here it is not 
necessary to specify the preliminary number of clusters (e.g., as in fc-means clustering). The correlation 
clustering problem formulation is motivated from documents/web pages clustering. This combinatorial 
model belongs to NP-complete class (e.g., |lll39l4np . 

Various versions of correlation problem formulations are examined: 

(a) weighted versions of the “correlation clustering functional” are considered as well (e.g., |94l95ll59] b 

(b) correlation clustering with partial information (e.g., [158) 1. 

(c) correlation clustering with noisy input (e.g., [423) 1. etc. 

Let us consider the weighted version of the problem. Let A = {Ai,Aj,An} be the initial set of 
elements. As a result, (n — 1)^ elements pairs can be considered: G = {gi, ■■■, g(n-i)^}- Each element of 
G corresponds to element pair (Ajj^,Aj^) and an element of proximity matrix Z = ||.Zjij 2 ll- Further, it is 
possible to replace scale [—1, -fl] for each edge (i.e., for each element from G or element of proximity matrix 
Z) by two quantitative scales for weights: negative quantitative (or ordinal) scale [—ui—, ...,0) instead of 
“—1” and positive quantitative scale (0,..., ui+] instead of “-I-1”. Evidently, element pair set is divided into 
two separated subsets G = G~ U G+ (without intersection, i.e., |G” p] G+| = 0) where Vg” € G~ weight 
estimate corresponds to negative quantitative scale above, where \/g~^ € G^ weight estimate corresponds 
to negative quantitative scale above. The clustering solution is: X = {Xi,..., A^,..., A^}. For this 
solution, two total quality parameters above are examined: 

(i) total agreements quality as (summarization by all intra-cluster pairs with positive edge weight) 
(5“®”(A) (maximization); 

(ii) total disagreements quality (summarization by all intra-cluster pairs with negative edge weight) 
(5*®“®”(A) (for minimization, by module). 

As a result, the weighted version of correlation clustering problem is (Fig. 3.16): 

Find clustering solution A such that: (i) Q°-s^{X) —>• max and (ii) |(5'^®®“®’'(A)| —min. 


( 0 , 0 ) 


Q“®''(A) 

Ideal 

point 


"Tx Ideal 


Trajectory of vector objective 
function (by algorithm iterations) 

(Q^*®“®XA),Q“®’'(A)) 


Constraint 
q > IQ^^^spA)! 


Qdzsagr^^X) 

Fig. 3.16. “Space” of solution quality 


Heuristics and approximation algorithms (e.g., PTAS) have been proposed for the problem versions 
(e.g., [33I39I40I662) '). Clearly, agglomerative (hierarchical) clustering scheme (i.e., selection of an element 
pair from set B for next joining for improvement of a current clustering solution) can be used here as a 
simple greedy algorithm (Bottom-Up process of selection of element pair with the best improvement of 
objective vector function and corresponding joining the elements), for example (Fig. 3.16): 


Stage 1. Calculation of the matrix of element pair V(A(ji), A(j 2 )), A(ji) e A, A(j 2 ) S A, ji ^ j 2 
proximities (“distances”). 
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Stage 2. Transformation of element pair proximities into positive (for similar elements) or negative 
(for dissimilar elements) weights (e.g., mapping). 

Stage 3. Specifying the initial clustering solution as composition of initial elements, vector objective 
function / = {X°), Q“9’'(X0)) = ( 0 , 0 ) (initial value, initial index 7 = 0 ). 

Stage 4- Searching for the element pair with the best improvement of vector objective function / (i.e., 
searching for Pareto-efficient point (s)). Integration of the corresponding both elements into a cluster 
or inclusion of the corresponding element into the cluster with the second element (i.e., new clustering 
solution) X‘> {q is parameter of algorithm iteration). Recalculation of the current value of vector objective 
function: f = (Q‘^*®“9’'(X^), Q“9'’(X'>')). 

Stage 5. If all elements are processed then GO TO Stage 7. 

Stage 6. Increasing index 7 = 7 -|- I, while constraint {X)\ < q is satisfied Go To Stage 4, 

else GO TO Stage 7. 

Stage 1. Stop. 

Complexity estimates of greedy heuristic above for two-objectives correlation clustering (by stages) are 
presented in Table 3.21. 


Table 3.21. Complexity estimates of stages for greedy agglomerative heuristic 


Stage 

Description 

Complexity estimate 
(running time) 

Stage 1. 

Calculation of distance matrix Z 

O(n^) 

Stage 2. 

Calculation of positive/negative weights 

O(n^) 

Stage 3. 

Specifying the initial solution 

0 ( 1 ) 

Stage 4. 

Searching for the best element pair 

(by Pareto-efficient improvement of objective function) 

O(n^) 

Stage 5. 

Analysis of algorithm end, recalculation of objective function 

0 (n) 

Stage 6 . 

Transition of computing process 

0 ( 1 ) 

Stage 7. 

Stopping 

0 ( 1 ) 


Example 3.4. The examined element set involves 11 elements: A = {Ai, ■■.,Aj, ...,Tii}. The weights 
of all edges are presented in Table 3.22. Here, two quantitative scales are used: [—6.5,0) and (0,3.5]. 


Table 3.22. Weights of edges {{Aj^, Aj^)} 


ji 

h ■ 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

1 


-3.0 

1.4 

- 0.6 

-5.1 

-5.5 

-3.5 

- 1.2 

-3.4 

-4.5 

-6.5 

2 



- 0.8 

-1.3 

-3.3 

-1.4 

3.1 

-2.9 

-3.6 

-5.1 

-4.9 

3 




- 1.1 

- 2.0 

-2.7 

- 2.1 

3.2 

-3.0 

-4.5 

-4.1 

4 





-0.5 

-3.7 

-2.5 

2.6 

-0.9 

-1.3 

- 2.2 

5 






-6.5 

-5.6 

- 1.1 

2.8 

-0.5 

- 6.1 

6 







3.5 

0.4 

- 1.8 

-3.2 

-0.3 

7 








0.5 

-0.3 

- 0.8 

2.9 

8 









1.0 

- 0.8 

- 2.8 

9 










3.0 

-5.5 

10 











- 6.0 


Initial information is the following (iteration index equals 7 = 0): (a) X'^ = {X^, ..., ..., where 

X? = {Hi}, X^ = {H2}, X3O = {H3}, Xl =_{H 4 }, X° = {A}, XI = {He}, X? = {H7}, X^ = {Hg}, 
Xg = {Tg}, X^g = {Tio}, X^^ = {All}; (b) / (X°) = (0,0); (c) improvement operations (i.e., inclusion 
of element (cluster) Aj into cluster XJ as Oj^i(Aj —>■ XJ (j = 1,10, l = j, 11) and corresponding 
improvements (by positive component or by negative component) of objective function / as A/(Oj_t) are 
presented in Table 3.23 (component 0 of the vector is not pointed out). 





54 


Table 3.23. Improvements of objective function A/(Oj^) (iteration index 7 = 0) 



Iteration 1. Selection of the best (Pareto-efficient) improvement operation Oe,? with the best im¬ 
provement A/(Oj_t) = (0,3.5). As a result, the following information is used for the next algorithm 
step: (a) = {Xl, X^, X^, Xl,} where X^ = {AJ, X^ = {A 2 }, A 3 I = {A 3 }, 

X 4 = {A 4 }, Xl = {A 5 }, Ay = {As, Ay}, Ag = {Ag}, A} = {Ag}, A}g = {Aio}, A}^ = {An}; (b) 
/ (A^) = (0,3.5); (c) improvement operations (i.e., inclusion of element/cluster A^j into cluster A^^) as 
Otj_t 2 (Atj Atj) and corresponding improvements (by positive component or by negative component) 
of objective function / as presented in Table 3.24 (component 0 of the vector is not pointed 

out). 


Table 3.24. Improvements of objective function A/(Oy^J (iteration index 7 = 1 ) 



Iteration 2. Selection of the best (Pareto-efficient) improvement operation Og^g with the best improve¬ 
ment A/(Oj,t) = (0,3.2). As a result, the following information is used for the next algorithm step: (a) 
A2 = {A}, A|, A|, A|, Ay2, A|, Ag^, Afg, Afy} where A} = {Ag}, A| = {Ag}, A|_= {A4}, A| = {Ag}, 
A| = {A6,A7}, A| = {A3,Ag}, A| = {Ag}, A^g = {Aig}, XI 4 = {All}; (b) f\x^) = (0,9.9); (c) 
improvement operations (i.e., inclusion of element/cluster A^^ into cluster X,^^) as —>■ A^^) and 

corresponding improvements (by positive component or by negative component) of objective function / 
as presented in Table 3.25 (component 0 of the vector is not pointed out). 


Table 3.25. Improvements of objective function A/(Oy^) (iteration index 7 = 2) 



corresponding Pareto-efHcient improvements are: A/( 02 , 7 ) = (—1.4,3.!) and A/(Ogpo) = (0,3.0). Op¬ 
eration Ogpo is selected. As a result, the following information is used for the next algorithm step: 
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(a) = {X!,XlXi,XlXlXi,X!„X!,} where X? = {Ai}, =^^ 2 }, = {A 4 }, = {A}, 

X 7 = {^ 6 ,^ 7 }, X| = {^3,218}, Xfo = {^Ig.Aio}, Xf^ = {All}; (b) f{x^) = (0,12.9); (c) improvement 
operations (i.e., inclusion of element/cluster into cluster X^^) as ^ 1 . 2 ) corresponding 

improvements (by positive component or by negative component) of objective function / as 
are presented in Table 3.26 (component 0 of the vector is not pointed out). 


Table 3.26. Improvements of objective function A/(Oj_t) (iteration index 7 = 3) 



A^2 • ^2 ^4 ^5 ^7 ^8 ^10 ^11 

A/ 

A7 = {Ag, A7} 
xi = {A3, As} 

= {M, Aio} 

-3.0 -0.6 -5.1 -9.0 (-1.2,1.4) -7.9 -6.5 

-1.3 -3.3 (-1.4,3.!) -3.7 -8.7 -4.9 

-0.5 -6.2 (-1.1,2.6) -2.2 -2.2 

-12.2 -3.1 (-0.5,2.8) -6.1 

(-4.8,0.9) -6.1 (-0.3,2.9) 

(-8.3,1.0) -2.8 

-8.8 


Iteration 4. Selection of the best (Pareto-efficient) improvement operations: 02,7 and 07 , 11 . The 
corresponding Pareto-efficient improvements are: A/(02,7) = (—1.4,3.!) and A /(07 . 11 ) = (-0.3, 2.9). 
Operation 07,11 is selected. As a result, the following information is used for the next algorithm step: 
(a) A4 = {Xt,Xi,XlXi,XlXfo,Xf,} where Xf = (Aij, Xl^= {A 2 }, X^ = {A4}, A| = {A5}, 
Xi = {^3,^8}, A/o = {A9,Aio}, Xf^ = {^ 6 ,^ 7 , All}; (b) T(X^) = (-0.3,15.8); (c) improvement 
operations (i.e., inclusion of element/cluster into cluster A^^) as Otj,t 2 (Atj A^^) and corresponding 

improvements (by positive component or by negative component) of objective function / as 
are presented in Table 3.27 (component 0 of the vector is not pointed out). 


Table 3.27. Improvements of objective function A/(Oj,t) (iteration index 7 = 4) 


^0 

A, 3 : X^ Xt X,^ A| A/g A/i = {A6,A7,Aii} 

A/ 

XI 

XI 

Xi 

Xi = {A3,As} 

= {Ag, Aio} 

-3.0 -0.6 -5.1 (-1.2,1.4) -7.9 -15.5 

-1.3 -3.3 -3.7 -8.7 (-6.3,3.!) 

-0.5 (-1.1,2.6) -2.2 -8.4 

-3.1 (-0.5,2.8) -18.2 

(-8.3,1.0) (-14.8,0.9) 

-17.6 


Iteration 5. Selection of the best (Pareto-efficient) improvement operations: 02,ii and 05 , 10 . The 
corresponding Pareto-efficient improvements are: A/(02,ii) = (—6.3, 3.1) and A/( 05 ,io) = (—0.5, 2.8). 
Operation 05,10 is selected. As a result, the following information is used for the next algorithm step: 
(a) A 5 = {Af,A|,A45,A|,Afo,Afi} where ^_i^= Mi}, = {Ag}, A| = {A4}, A| = {A3,A8}, 
Afg = {AojAqjAio}, All = {A 6 ,A 7 ,Aii}; (b) / (A®) = (—0.8,18.6); (c) improvement operations (i.e., 
inclusion of element/cluster A^^ into cluster A^^) as 0^,13 (A^^ A^^) and corresponding improvements 

(by positive component or by negative component) of objective function / as A/( 0 ^, 13 ) are presented 
in Table 3.28 (component 0 of the vector is not pointed out). 


Table 3.28. Improvements of objective function A/(Oj,t) (iteration index 7 = 5) 


^0 

A,3 : A| a/ A| A/g A/i = {A6,A7,An} 

A/ 

X! 

Xg = {A 3 , As} 

-^10 = {A5,A9,Aig} 

-3.0 -0.6 (-1.2,1.4) -13.0 -15.5 

-1.3 -3.7 -12.0 (-6.3,3.!) 

(-1.1,2.6) -2.7 -8.4 

(-11.4,1.0) (-14.8,0.9) 

-35.8 


Iteration 6. Selection of the best (Pareto-efficient) improvement operations: 02,ii and 04 , 8 . The 
corresponding Pareto-efficient improvements are: A/(02 11 ) = (-6.3,3.!) and Af{Oi^s) = (-1.1,2.6). 
Operation C> 4,8 is selected. As a result, the following information is used for the next algorithm step: (a) 
A6 = {a6,A|, A|,A6o,A6i} where Af = {AJ, A| = {A 2 }, A| = {A3,A4,A8}, Afg = {A5,A9,Aio}, 






56 


Xfi = {Aq,Ay,Aii}; (b) /^(X®) = (—1.9,21.2); (c) improvement operations (i.e., inclusion of ele¬ 
ment/cluster into cluster as ^i. 2 ) corresponding improvements (by positive 

component or by negative component) of objective function / as A/(Otj ^ 2 ) ^^e presented in Table 3.29 
(component 0 of the vector is not pointed out). 


Table 3.29. Improvements of objective function A/(Oj^) (iteration index 7 = 6) 



X,2: X® Xg® X®o X®i = {A6,A7,An} 

X® 

-^8 ~ {^3)A4,Ag} 

^10 = {^ 5 ,^ 9 , Alo} 

-3.0 (-1.8,1.4) -13.0 -15.5 

-5.7 -12.0 (-6.3,3.!) 

(-14.1,1.0) (-20.1,0.9) 

-35.8 


Iteration 7. Selection of the best (Pareto-efficient) improvement operations: 02 ,ii and Oi, 8 - The 
corresponding Pareto-efficient improvements are: A/( 02 ,ii) = (—6.3,3.!) and Af{Oi^s) = (—1-8,1.4). 
Operation is selected. As a result, the following information is used for the next algorithm step: 
(a) X7 = {X^XlXJo^Xj,} where Xj = {A 2 }, X| = {Ai, A 3 , A 4 , Ag}, Xj^ = {A 5 ,A 9 ,Aio}, X^, = 
{A 6 ,A 7 ,Aii}; (b) / (X^) = (—3.7,22.6); (c) improvement operations (i.e., inclusion of element/cluster 
Xtj into cluster X^^) as On^t 2 (^i.i ^ 1 . 2 ) and corresponding improvements (by positive component or 
by negative component) of objective function / as A/(On,t 2 ) are presented in Table 3.30 (component 0 
of the vector is not pointed out). 


Table 3.30. Improvements of objective function A/(Oj^) (iteration index 7 = 7) 



X,2: X/ X'o X/i = {A6,A7,Aii| 

-^8 = {^ 1 ) ^3, A 4 , Ag} 
= {^5, Ag, Aio} 

-8.0 -12.0 (-6.3,3.!) 

(-27.1,1.0) (-35.6,0.9) 

-35.8 


Iteration 8. Selection of the best (Pareto-efficient) improvement operation 02 ,ii- The corresponding 
Pareto-efficient improvement is: A/(C> 2 ,ii) = (—6.3,3.1). As a result, the following information is used for 
the next algorithm step: (a) X® = {X|, Xfg, Xf;^} where X| = {Ai, A 3 , A 4 , Ag}, Xfg = {A 5 ,A 9 ,Aio}, 
Xf-^ = {A 2 , Ag, Ay, All}; (b) / (X®) = (—10.0,25.7); (c) improvement operations (i.e., inclusion of 
element/cluster X^^ into cluster X,^^) as Ou^^L^iXui —^ and corresponding improvements (by positive 

component or by negative component) of objective function / as A/(Otj_t 2 ) are presented in Table 3.31 
(component 0 of the vector is not pointed out). 

Table 3.31. Improvements of objective function A/(Oj^) (iteration index 7 = 8) 


^0 

X, 2 : X« X«o X«i = {A6,A7,An} 

^8 = {Ai, A 3 , A 4 , Agj 
Alfo = {^ 5 , Ag, Aio} 

(-27.1,1.0) (-43.8,0.9) 

-47.8 


Finally, it is reasonable to consider the result of iteration 8 as the clustering solution: 

V _ vS _ / v8 vS vS \ 

A—A — |Ag , A]^q, 

Table 3.32 contains a list of main research directions in correlation clustering. 

On the other hand, it is possible to use a multiset based problem formulation. It is possible to replace 
scale [—1,4-1] (or two quantitative scales above) for each edge (i.e., for each element from G or element 
of proximity matrix Z) by two ordinal scales: negative ordinal scale [—A:“,..., —1] instead of “—1” and 
positive ordinal scale [-fl,..., fc+] instead of “4-1”. Note, calculation of edge weights upon the above- 
mentioned scales is sufficiently easy (e.g., mapping of the quantitative estimate into the ordinal scale). 
For the clustering solution X = {Xi,..., X^j two total quality parameters can be calculated as follows: (i) 
total agreements quality as multiset estimate (summarization by the component for all intra-cluster pairs 
with positive edge weight) Q°‘^''{X) (maximization); (ii) total disagreements quality as multiset estimate 
(summarization by the component for all intra-cluster pairs with negative edge weight) ( 3 ‘^*^“®'’(X) (for 
minimization). As a result, the multiset based correlation clustering problem is (Fig. 3.17): 

Find clustering solution X such that Q°‘^^(X) max and |Q‘^*^“®’^(X)| - 5 - min. 
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Table 3.32. Correlation clustering 


No. 

Research direction 

Source(s) 

1 . 

Basic problem formulations and complexity 

I11l,33l.39l4()l52l345lbb2l 

2 . 

Surveys 

11 1 140134516621 

3. 

Comparing methods for correlation clustering 

m 

4. 

Approximation algorithms (including PTAS) 

I33l39l4()l225l662l 

5. 

Weighted versions of correlation clustering problems 

I94I95I159I 

6 . 

Correlation clustering with fixed number of clusters 

fn5\ 

7. 

Maximizing agreements via semidefinite programming 

15641 

8 . 

Minimizing disagreements on arbitrary weighted graphs 

m 

9. 

Global correlation clustering 

m 

10 . 

Correlation clustering with partial information 

m 

11 . 

Correlation clustering with noisy input 

|42^ 

12 . 

Error bounds for correlation clustering 

j3n8j 

13. 

Robust correlation clustering 

I4I344I 

14. 

Correlation clustering in image segmentation 

m 



(a) disagreements (b) agreements 

quality quality 

Fig. 3.17. Multiset quality posets (lattices) 


3.5.4. Network communities based clustering 

In recent decades, “network communities based clustering” as a new research direction has been orga¬ 
nized (e.g., |206l226l285l370l457l458l459l460l461l496j '). The largest connected components are examined 
as “network communities”, for example: cliques, quasi-cliques, cliques/quasi-cliques with leaves, chains 
of cliques/quasi-cliques, integrated groups of clisues/quasi-cliques (Fig. 3.18). 


Community 1 
(clique) 


Community 3 
(quasi-clique) 


Community 4 
(chain of clique, 
quasi-clique) 



Community 'i 
(clique with leave) 

Fig. 3.18. Illustration for network communities 


Community 5 
(quasi-clique with leave) 


Community 6 (g^roup 
of cliques/quasi-cliques) 


The network example in Fig. 3.18 does not contains overlaps (i.e., without intersection of community 
structures). Fig. 3.19 illustrates the overlaps. 
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(a) Sparse overlaps 



(b) Dense overlaps 


Fig. 3.19. Illustration for overlaps in structures 


The detection of “network communities structures” corresponds to complex combinatorial optimization 
models (e.g., linear/nonlinear integer programming, mixed integer programming). The models belong to 
NP-hard problems (e.g., |73ll27l206l460) b Table 3.33 contains a list of basic research directions in 
community network based clustering. 

A list of basic algorithmic approaches for finding communities involves the following (e.g., [20614591 
14601461] '): (i) graph partitioning (e.g., minimum-cut method), (ii) hierarchical clustering (greedy agglom- 
erative algorithms), (iii) Girvan-Newman algorithm (edge betweeness), (iv) modularity maximization 
approaches, (v) spectral clustering methods, (vi) methods based on statistical inference, (vii) clique 
based methods. 

Modularity of a graph can be defined as a normalized tradeoff between edges covered by clusters and 
squared cluster degree sums [731461] . The problem is formulated as combinatorial optimization model. 
For the modularity maximization, several main algorithms are pointed out [73] : (a) greedy agglomeration 
[1271457] , (b) spectral division [4591610] . (c) simulated annealing [2491504] . (d) extremal optimization 
[178] . An example of modularity algorithm as greedy agglomerative heuristic is the following [457] : 


Stage 1. Trivial clustering: each node corresponds to its own cluster. 

Stage 2. Cycle by cluster pairs: 

Stage 2.1. Calculation of possible increase of modularity for merging each cluster pairs. 

Stage 2.2. Merging the two clusters with maximum possible increase. 

Stage 2.3. If increasing of modularity by merges of cluster pair is impossible then GO TO Stage 3. 
Stage 2.4. Go To Stage 2.2. 

Stage 3. Stop. 

In this algorithm, algorithmic complexity estimate equals 0{{p + n)n) or 0(n^) [457] . 

The general scheme of Girvan-Newman (GN) algorithm based on edge betweenness is [226] : 


Step 1. Calculation of the betweenness score for each the edges. 

Step 2. Deletion of the edges with the highest score. 

Step 3. Performance analysis for the network’s components. 

Step 4- If all edges are deleted and the system breaks up into N non-connected nodes Go TO Step 5. 
Otherwise GO TO Step 1. 

Step 5. Stop. 

Algorithmic complexity estimate of the algorithm equals 0{p^n) {p is the number of edges) [226] '). 
3.6. Towards fast clustering 

Many applications based on very large data sets/networks require fast clustering approaches (e.g., [1271 
I457I546I585I600I662] !. In Table 3.34, basic ideas for fast clustering schemes are pointed out. Generally, 
many fast clustering schemes consist of two basic levels (global level and local level): (a) partition of 
the initial problems into local problems (i.e., decreased dimension, limited type of objects/elements) 
(global level), (b) clustering of local clustering problems (local level), (c) composition/integration of local 
clustering solutions into a resultant global clustering solution (global level). 

In Table 3.35, a list of basic fast local clustering algorithms (i.e., fast sub-algorithms) is presented. 
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Table 3.33. Community network based clustering 


No. 

Research direction 

Source(s) 

1 . 

Basic issues: 


1 .1. 

Basic problem formulations 

17312061226137014611459146014961608163516361 

1 .2. 

Basic surveys 

1731611206I370I439I4611459I460I496| 

1.3. 

Problems complexity 

|73I127I206I460| 

1.4. 

Overlapping (fuzzy) community structures 

I230l608l622l633l634l635| 

1.5. 

Analysis/evaluation of community structures 

|370l46ll608l636| 

2 . 

Main algorithms/solving schemes: 


2 .1. 

Algorithm based on edge betweeness 
(divisive algorithm) 

[226] 

2 .2. 

Modularity algorithm as greedy agglomerative 
heuristic 

|457I475| 

2.3. 

“Karate Club” algorithm 

m 

2.4. 

Kernighan-Lin method and variants 


2.5. 

Overlapping communities (clique percolation, 
local expansion, dynamic algorithms, etc.) 

|230I622I633I634I635| 

2 .6. 

Spectral clustering algorithms, modifications 


2.7. 

Genetic algorithms 

l405i 

2 .8. 

Agent-based algorithms 

|250| 

3. 

Modularity clustering (maximum modularity): 


3.1. 

Surveys 

|73I459I664|636| 

3.2. 

Tripartite modularity (three vertex types) 

i44§|449j 

3.3. 

Modularity in fc-partite networks 

|406| 

3.4. 

Greedy agglomeration algorithm 

I127I457I 

3.5. 

Spectral division algorithm 

I459I610I 

3.6. 

Simulated annealing algorithms 

|249I504| 

3.7. 

Detecting communities by merging cliques 

m 

3.8. 

Extremal optimization scheme 
(mathematical programming) 

[7|T7^ 

3.9. 

Global optimization approach 

m 

3.10. 

Memetic algorithm 

[^ 

3.11. 

Random works algorithms 

[4951 

3.12. 

Multi-level algorithms 


4. 

Large networks: 


4.1. 

Communities in large networks 

1591127I230I285I286I370I495I636I 

4.2. 

Communities in mega-scale networks 

|600| 

4.3. 

Communities in super-scale networks 

m 

4.4. 

Tracking evolving communities in large networks 

|2M| 

5. 

Applications: 


5.1. 

World Wide Web 

I168I370I448I 

5.2. 

Journal/article networks, citation networks, etc. 

II0612061516I 

5.3. 

Social networks (friendship, collaboration, etc.) 

|206I226I248I457I460I461|600|636| 

5.4. 

Biological networks 

120612261461 1 

5.5. 

Purchasing network 

m 

5.6. 

CAD applications 

m 

5.7. 

Antenna-To-Antenna network 
(mobile phone network) 

I60I407I 
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Table 3.34. Main approaches to fast clustering 


No. 

Approach 

Solving schemes 

Source(s) 

1 . 

Aggregation of object/network 
nodes 

Hierarchical clustering (Bottom-Up, 
step-by-step node aggregation) 

I3()()l3()2l546l 

2 . 

Division of objects/network 
nodes (partition/decomposition); 

Top-Down scheme 


2 .1. 

Pruning of objects/network 
nodes (Fig. 3.20) 

1 .Selection of basic edge betweenness 
in graph and decoupling 
(Top-Down scheme) 

2 .Clustering in each graph part 
(if needed) 

I226I546I 

2 .2. 

Multi-level schemes (partition, 

1 .Partition of object set/network 

|584I585I662| 


clustering, integration of solutions): 

2 . Clustering of local regions 

3. Composition of local solutions 

this paper 

2 .2.1. 

“Basic” objects (special “key” 
objects/nodes) based clustering 
(Fig. 3.21) 

1 . Detection of “basic” objects/nodes 
(e.g., by filtering) 

2 . Clustering of “basic” objects/nodes, 

3. Joining other elements/nodes 
to obtained clusters 

this paper 

2 .2.2. 

Grid-based clustering 

Dividing the space into cells 

1345140216621 

2.2.3. 

Grid-based clustering in data streams 

Online clustering of data streams 

|411I486| 

2.2.4. 

Grid-based clustering (composition): 
multiple division of objects 
“space”/network into cells/regions 
(e.g., axis-parallel subspaces), 
region-based clustering, 
composition of local solutions 
(Fig. 3.22) 

1. Grid over object “space”/network 

2 . Analysis of grid regions 

3.Selection of “non-empty” regions 

4. Clustering in “dense” regions 

5. Clustering in “sparse” regions 
(while taking into account 
solutions in “dense” regions) 

6 . Composition of regions solutions 

this paper 

2.2.5. 

Grid-based clustering (extension): 
multiple division of objects 
“space”/network and “extension” 
of clustering solutions 
(with condensing of clusters, 
as in dynamic programming) 

(Fig. 3.22) 

1. Grid over object “space”/network 

2 . Analysis of grid regions 

3.Selection of “non-empty” regions 

4. Clustering in “dense” regions 

5. Extension of “dense” regions 

by neighbor region(s) and extension 
of clustering solution(s) 

this paper 

2 .2.6. 

Division of object “space” / 

1 .Detection of objects by types 

I406I448I449I 


network by types (fc-partite 
network) (close to 2.2.1) 

2 . Clustering for each part 

3. Composition of clustering solutions 

this paper 

3. 

Gomposite (multistage, concurrent. 

Composition/combination 

m 


multi-techniques) approaches 

of various approaches 

this paper 



Fig. 3.20. Edge betweenness for decoupling 
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Clusterinff solution 
for “basin’ objects 


Final clustering 
solution 




Clus¬ 

tering 



of 



“basic” 

objects 



VQ 


Joining 

of other 
objects 

to /‘basic” 
object 

clusters 


Fig. 3.21. “Basic” objects based clustering framework 


o© 



Fig. 3.22. “Grid” over object “space”/network 


Table 3.35. List of some fast local clustering algorithms 


No. 

Fast scheme 

Description 

Complexity 
estimate 
(running time) 

Source(s) 

1 . 

Basic agglomerative 
(hierarchical) algorithm 

Bottom-up joining the 
closest object pair 

0 (/?) 

131)2] 

2 . 

Balanced by cluster size 
hierarchical algorithm 

Bottom-up joining the 
closest object pair under 
constraints for cluster size 

0(n3) 


3. 

Minimum spanning tree 
based algorithm 

Clustering the spanning 
tree nodes 

0 (n log n) 

[23512441445] 

|479I492I555| 

I6()6l626l66()i 

4. 

Balanced by cluster size 
minimum spanning tree 
based algorithm 

Clustering the spanning 
tree nodes under 
constraints for cluster size 

0 (n logn) 


5. 

Graph clustering 
algorithm 

Detection of network 
communities (edge 
betweeness of the graph) 

0 {p^n) 

[22^ 

6 . 

Modularity graph 
clustering algorithm 

Modularity based detection 
of network communities 

0 {(p -I- n)n) or 
O(n^) 

[457] 

7. 

Algorihtms based on 
grid over “space of 
object coordinates” 
(partition space 
techniques) 

Assignment of objects into 
local regions of “space 
of object coordinates” 

0(n + n' X n") 

[n' <g; n, n" <g; n) 

P5] 

8 . 

Clustering based on 
cores decomposition 
of networks 

Preliminary cores 
decomposition 
of covering graph 

0 {n?) -b 0 {h) 

m 
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4. Some Combinatorial Clustering Problems 

4.1. Clustering with interval multiset estimates 

4.1.1. Some combinatorial optimization problems with multiset estimates 

Multiset estimates are a simplification of multicriteria (vector) estimates. As a result, a simple scale (a 
little more complicated as an ordinal scale) is used. On the other hand, multiset estimate is a simple gen¬ 
eralization of well-known binary voting procedure. Thus, multiset estimate can be used for simplification 
of multicriteria (multi-parameter) measurement in various problems/procedures. 

In |386l392j . basic operations over multiset estimates have been described: integration, vector-like 
proximity, aggregation, and alignment. 

Integration of estimates (mainly, for composite systems) can be considered as summarization of the 
estimates by components (i.e., positions). 

Let us consider vector-like proximity of two multiset estimates |386l392j . Let Ai and A 2 be two al¬ 
ternatives with corresponding interval multiset estimates e(Ai), 0 (^ 2 ). Vector-like proximity for the 
alternatives above is: 5 {e{Ai),e{A2)) = (<5“(Ai, A 2 ), i5+(Ai, A 2 )), where vector components are: (i) 5~ 
is the number of one-step changes: element of quality t -I- 1 into element of quality i (t = 1, / — 1) 
(this corresponds to “improvement”); (ii) (5^ is the number of one-step changes: element of qual¬ 
ity 6 into element of quality t -I- 1 (t = 1,Z — 1) (this corresponds to “degradation”). It is assumed: 
|5(e(Ai),e(A2))| = |r(Ai,A2)| + |5+(Ai, ^ 2 )!. 

Aggregation of multiset estimates can be defined as a median estimate for the specified set of initial 
estimates (traditional approach). Let E = {ci,..., e^,..., Cn} be the set of specified estimates (or a 
corresponding set of specified alternatives), let D be the set of all possible estimates (or a corresponding 
set of possible alternatives) {E C D). Thus, the median estimates (“generalized median” and “set 
median” M'*) are: = argminMez? M®=argminMg£ YZ=i |<5(M,eK)|. 


Multiple choice problem with multiset estimates can be considered as follows [3861392] . Basic 
multiple choice problem is: (e.g., |218l325j b 

m qi m Qi qt 

max EE C-ijXij S.t. EE b; ^ Xij <1, i = 1, m; Xij G {0,1}. 

j=l i=l j=l j=l 

In the case of multiset estimates of item “utility” ei,f G {1,..., f,..., m} (instead of c^), the following 
aggregated multiset estimate can be used for the objective function (“maximization”) [3861392] 1: (a) an 
aggregated multiset estimate as the “generalized median”, (b) an aggregated multiset estimate as the 
“set median”, and (c) an integrated multiset estimate. 

A special case of multiple choice problem is considered: 

(1) multiset estimates of item “profit”/“utility” dj, i G {I, ...,i, = l,qi (instead of aj), 

( 2 ) an aggregated multiset estimate as the “generalized median” (or “set median”) is used for the 
objective function (“maximization”). 

The item set is: A = U™^Ai, = {(f, 1), (z, 2), ...,(*, g^)}. 

Boolean variable Xij corresponds to selection of the item The solution is a subset of the initial 

item set: S = {(i,j)\xij = 1}. The problem is: 

max e{S) = max M = arg min E IWe..,)l, 

iiJ)eS={{i,j)\xij=l} 


m qi 

S.t. ^ ^ — ^5 

i=l j=l 


Qi 

^ ^ Xjj — 1, Xij G {0, 1}. 
i=i 


Here the following algorithms can be used (as for basic multiple choice problem) (e.g., [218132513861 
m\)- (i) enumerative methods including dynamic programming approach, (ii) heuristics (e.g, greedy 
algorithms), (iii) approximation schemes (e.g., modifications of dynamic programming approach). 


Combinatorial synthesis (Hierarchical Multicriteria Morphological Design - HMMD) with ordinal 
estimates of design alternatives is examined as follows f [374l376l385l386l392] L A composite (modular. 




















63 


decomposable) system consists of components and their interconnection or compatibility (IC). Basic 
assumptions of HMMD are the following: (a) a tree-like structure of the system; (b) a, composite 
estimate for system quality that integrates components (subsystems, parts) qualities and qualities of IC 
(compatibility) across subsystems; (c) monotonic criteria for the system and its components; (d) quality 
of system components and IC are evaluated on the basis of coordinated ordinal scales. The designations 
are: (1) design alternatives (DAs) for leaf nodes of the model; (2) priorities of DAs {i = 1, Z; 1 corresponds 
to the best one); (3) ordinal compatibility for each pair of DAs {w = 1, z/; v corresponds to the best one). 

Let S' be a system consisting of m parts (components): i?(l), ...,R{i), ...,R{m). A set of design alter¬ 
natives is generated for each system part above. The problem is: 

Find a composite design alternative S = S(l)*...*S(j)*...*S(m) of DAs (one representative design 
alternative S(i) for each system component/part R{i), i = l,m ) with non-zero compatibility between 
design alternatives. 

A discrete “space” of the system excellence (a poset) on the basis of the following vector is used: 
N{S) = {w{S)]e{S)), where w{S) is the minimum of pairwise compatibility between DAs which cor¬ 
respond to different system components (i.e., V and Ajj, 1 < j 2 < rn) in S, e(S') = 

(rji, ..., ..., ? 7 /), where is the number of DAs of the tth quality in S. Further, the problem is described 

as follows: 

max e{S), max w{S), s.t. w{S) > 1. 

Here, composite solutions which are nondominated by N{S) (i.e., Pareto-efHcient) are searched for. 
“Maximization” of e{S) is based on the corresponding poset. The considered combinatorial problem is 
NP-hard and an enumerative solving scheme is used. 

Here, combinatorial synthesis is based on usage of multiset estimates of design alternatives for system 
parts. For the resultant system S = S'(l) * ... * S{i) * ... * S{m) the same type of the multiset estimate 
is examined: an aggregated estimate (“generalized median”) of corresponding multiset estimates of its 
components (i.e., selected DAs). Thus, N{S) = {w{S);e{S)), where e{S) is the “generalized median” of 
estimates of the solution components. Finally, the modified problem is: 

m 

max e(S) = = arg min \5(M, e(Si))\, max w(S), s.t. w(S) > 1. 

mgd ^' 

i=l 

Here enumeration methods or heuristics are used (e.g., [37413761385138613^ 1. 

Assignment problem with multiset estimates is formulated as follows. Estimates of “profits” / “utilities” 
of local assignments (i.e., item-position) {cij} can be replaced by multiset estimates {e^}. Further, 
summarization in objective function can be implemented as summarization of multiset estimates or by 
searching for a median estimate {S is an assignment solution for all elements i = l,n): 

max e{S) = max M = arg min 

m n 

s.t. < 1, j = l,n-, E! — Ip = 1, w; Xjj g {0,1}, i = l,m, j = 1, n. 

i=i j=i 

In the case of generalized problem (e.g., it is possible to assign several items to each position), the problem 
is (i.e., change of constraint for each position j): 

max e(S) = max M = arg min > |(5(M, Dl, 

MeD ^ 

m n 

s.t. — 1)* = 1)^; ^i,j £ {0, 1 }d = = 1 )^) 

i=i j=i 

here bj is constraint for number of assigned elements for each position j). Clearly, other analogical 
constraints for each positions can be used as well (i.e., by other types of resources). It is reasonable to 
use heuristics as solving schemes. 
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4.1.2. Towards Clustering with interval multiset estimates 

In agglomerative algorithms, the basic methodological problem consists in selection/design of proximity 
measure for objects/clusters. Evidently, the measure is often based on many parameters and it is necessary 
to use vector proximity/distance. Here, it may be reasonable to simplify of the solving procedure via 
transformation of the vector proximity (proximities) into multiset estimate(s). Minimization of multiset 
estimates is a simple process (in some complex situations Pareto-efficient point(s) can be used). 

Analogical approach can be used in k-means clustering method by the usage of interval multiset es¬ 
timates instead of proximity/distance of objects to cluster centroids. In the case of assignment based 
clustering, the above-mentioned model based on multiset estimates can be used. 

Example 4.1. A simplified numerical example is based on data from example 3.2: set of 9 end users 
{A = {1, ...,i, ...,9}) (Fig. 4.1) and their quantitative estimates (Table 3.15, vector estimate Xi,yi,Zi). 



Table 4.1 contains pair vector proximity estimates: D{ii,i 2 ) = (da:(u, * 2 ), dy(H, * 2 ), dz(H, * 2 )), where 

dx (H , ^ 2 ) — \^ii ^12 I; dy (^ 7 ^ 2 ) — \yii yi2 15 dz {AA 2 ) — \^ii I ■ 


Table 4.1. Vector quantitative proximity between end users £)(ii,* 2 ) 



Table 4.2 contains corresponding vector ordinal proximity r{ii,i 2 ) = (?’a:(*ij*2),"Cy (H, *2),?’z(h,*2)) 
(ordinal scale [1, 2, 3] is used, 1 corresponds to close values). Ordinal values are calculated as follows (for 
parameter x, for other parameters calculation is analogical): 


1 

f 

if 

jmin 

^x 

< 

dxiii,i2)< dr" + % 

3 ’ 

rx{.iiA2) = < 

2 , 

if 

jmin 

^x 

+ 

^ < dx(*i,i2) 

< 

1 

1 3, 

if 

jmin 

^x 

+ 

2 ^ < dx{ii,i2) 

< -f , 

where 








^min _ rY^^■n - X 

d^ix), d™" 

= miui^Adiiy), 

= 

minjg^ di{z 


^ niax,g^ d^(x), = max^g^ d,{y), = max,g^ di{z); 

A _ Amax _ ^min A _ ^max _ ^min A _ ^max _ ^min 

l_^X — , L^y — Uy Uy ^ L^z — ' 

Table 4.3 contains corresponding multiset estimates. The used multiset scale (poset) is shown in Fig. 
4.2 (assessment over scale [1,3] with three elements; analogue of poset in Fig. 2.16). 
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Table 4.2. Vector ordinal proximity between end users r{ii,i 2 ) 




( (3,0,0)) 

( ( 2 , 1 , 0 )) 



( ( 2 , 0 , 1 )) ( ( 1 , 2 , 0 )) 



( (1,1,1)) ( (0,3,0)) 



C (1,0,2)) C (0,2,1)) 



( ( 0 , 1 , 2 )) 

C (0,0,3)) 

Fig. 4.2. Multiset scale 

Application of hierarchical agglomerative balance by cluster size (< 3) algorithm (while taking into 
account multiset proximity estimates) leads to the following results (integration of multiset estimates is 
implemented as searching for the median estimate). 

Iteration 1. The smallest pair multiset proximity (3,0,0) correspond to the following item node pairs 
(with next integration of the pairs): (1,2) => Ji^ 2 , (4,5) ^ J 4 , 5 , and (7,8) (concurrent elements 

integration). 

Iteration 2. The smallest pair multiset proximity (2,1,0) correspond to the following item node pair 
(with next integration of the pairs): Jr,8, 9 - Cluster Xi = {7,8,9} is obtained. 

Iteration 3. The smallest pair multiset proximity (2,0,1) correspond to the following item node pair 
(with next integration of the pairs): (J4,5,3) =J> J 3 , 4 , 5 . Cluster X 2 = {3,4,5} is obtained. 
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Finally, the following clustering solution can be considered: 

X = {Xi,X2,X3,X4,X5}, = {7,8,9}, X 2 = {3,4,5}, X 3 = {1,2}, X 4 = { 6 }. 

Note, a numerical example for assignment based clustering can be considered analogically (e.g., on the 
basis of data from example 3.2). 

4.2. Restructuring in clustering 

Restructuring approach in combinatorial optimization has been suggested by the author in [3811392) . 
In this section, restructuring approach for clustering problems is briefly described. 

4.2.1. One-stage restructuring 

Fig. 4.3 and Fig. 4.4 illustrate the restructuring process fone-stage framework! [3811392) . Restructuring 
in clustering problem is depicted in Fig. 4.5. 



On T2 t 

Fig. 4.3. Framework of restructuring process [3811392) 
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(t = ri) 


Fig. 4.4. Restructuring scheme [3811392] 


Let P be a combinatorial optimization problem with a solution as structure S (i.e., subset, graph), 
O be initial data (elements, element parameters, etc.), f{P) be objective function(s). Thus S'(O) be 
a solution for initial data O, f{S{ft)) be the corresponding objective function. Let be initial data 
at an initial stage, /(S'(O^)) be the corresponding objective function. be initial data at next stage, 
/(S'(02)) be the corresponding objective function. As a result, the following solutions can be considered: 
(a) = S'(n^) with f{S{n^)) and (b) with /(S'(n^)). 

In addition it is reasonable to examine a cost of changing a solution into another one: H{S°^ —^ S^). 
Let p{S°‘,S^) be a proximity between solutions S'" and S^, for example, p{S°‘,S^) = |/(S") — /(S^)|. 
Clearly, function /(S) can be a vector function. Thus, the following version of restructuring problem is 
considered: 


Find a solution S* while taking into account the following: 

(i) H{S^ —>• S*) —>■ min, (ii) p(S*,S^) —>■ min (or constraint). 
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n 


T2 


t 


Fig. 4.5. Restructuring in clustering problem 


The basic optimization model can be examined as the following: 
minp(5'*,S'2) s.t. H{S^ ^ S^) <h, 

where /i is a constraint for cost of the solution change. In a simple case, this problem can be formulated 
as knapsack problem for selection of a subset of change operations [3811392] : 

n n 

max '^clx^ s.t. Xie{ 0,l}. 

i=l i=l 

In the case of interconnections between change operations, it is reasonable to consider combinatorial 
synthesis problem (i.e., while taking into account compatibility between the operations). 


Example 4.2. Initial information involves the following: 

(i) set of elements A = {1, 2, 3,4, 5, 6, 7, 8,9}; 

(ii) initial solution 1 (t = n): {Xl,X 2 ,X^}, clusters Xl = {1,3,8}, X} = 

(iii) solution 2 {t = T2): = {Xf, X}, X|}, clusters X^ = {2, 3}, X| = {5, 7 

(v) general set of considered possible change operations (each element can be 

solution clusters is not changed): 

On: none, O12: deletion of element 1 from cluster X^, addition of element 
deletion of element 1 from cluster X^, addition of element 1 into cluster X^; 

O21: none, O22: deletion of element 2 from cluster X^, addition of element 
deletion of element 2 from cluster X^, addition of element 2 into cluster X^; 

O31: none, O32: deletion of element 3 from cluster X^, addition of element 
deletion of element 3 from cluster X^, addition of element 3 into cluster X^; 

O41: none, O42: deletion of element 4 from cluster X^, addition of element 
deletion of element 4 from cluster X^, addition of element 4 into cluster X^; 

O51: none, O52: deletion of element 5 from cluster X^, addition of element 
deletion of element 5 from cluster X^, addition of element 5 into cluster X^; 

Oei: none, 032: deletion of element 6 from cluster X^, addition of element 
deletion of element 6 from cluster X^, addition of element 6 into cluster X^; 

O71: none, O72: deletion of element 7 from cluster X^, addition of element 
deletion of element 7 from cluster X^, addition of element 7 into cluster X^; 

Osi- none, Os 2 - deletion of element 8 from cluster X^, addition of element 
deletion of element 8 from cluster X^, addition of element 8 into cluster X^; 

O91: none, O92: deletion of element 9 from cluster X^, addition of element 
deletion of element 9 from cluster X^, addition of element 9 into cluster X^. 

In this case, optimization model (multiple choice problem) is: 


{2,4,7},X3i = {5,6,9}; 
,8}, X| = {1,4,6,9}; 
replaced, the number of 

1 into cluster X^, O13: 

2 into cluster X^, O23: 

3 into cluster X^; O33: 

4 into cluster X^, O43: 

5 into cluster X^, O53: 

6 into cluster X^, 

7 into cluster X^, O73: 

8 into cluster X^, Os 3 '- 

9 into cluster X^, O93: 


n 3 

E E 

i=i j=i 


S.t. 


2=1 


3 

E 

j=i 


^ 6, Xij G {O; 1}; 


max 
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where a{Oij) is the cost of operation Oij, c{Oij) is a “local” profit of operation Oij as influence on 
closeness of obtained solution X* to clustering solution X'^. Generally, it is necessary to examine quality 
parameters of clustering solution as basis for objective function(s). 

Evidently, the compressed set of change operations can be analyzed: 

Oi: deletion of element 1 from cluster addition of element 1 into cluster X^; 

02 - deletion of element 2 from cluster X^, addition of element 2 into cluster X^; 

O 3 : deletion of element 4 from cluster X^, addition of element 4 into cluster X^; 

O 4 : deletion of element 5 from cluster X^, addition of element 5 into cluster X^; 

O5: deletion of element 8 from cluster X^, addition of element 8 into cluster X^. 

In this case, optimization model is knapsack problem: 


9 

E' 

i=i 


'^c{Oj)xj s.t. '^a{Oj)xj < b, a:j G{0,l}, 
i=i 


where a{Oj) is the cost of operation Oj, c(Oj) is a “local” profit of operation Oj as influence on closeness 
of obtained solution X* to clustering solution X^. 

Finally, let us point out an illustrative example of clustering solution (Fig. 4.6): 

X* {Xi*,X2*,X3*}, clusters XJ = {1,2,3}, X^* = {7,8}, X3* = {4,5,6,9}. 
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Fig. 4.6. Example: restructuring of clustering solution 

4.2.2. Multistage restructuring, cluster/element trajectories 

This kind of clustering (or classification) model/problem is close to multistage system design [20612861 
I389I391I392) . Fig. 4.7 and Fig. 4.8 illustrate multistage classification and multistage clustering problems: 
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Fig. 4.7. Illustration of multistage classification 
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1. Multistage classification (Fig. 4.7): the same set of classes at each time stage (here: four classes 
L^, L^), elements can belong to different classes at each stage. Here: elements 1, 2, 3; trajectory for 
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element 1: J(l) =< L^,L^,L^ >, trajectory for element 2: J(2) =< L^,L^,L? >, trajectory for element 
3: J(3) =< L3,L4 

2. Multistage clustering (Fig. 4.8): different set of clusters at each time stage can be examined, 
elements can belong to different clusters at each stage. Here: elements 1, 2, 3; trajectory for element 
1: J(l) =< L\,L\,L\ >, trajectory for element 2: J(2) =< L\,L\,L1 >, trajectory for element 3: 
J(3) =<LlLiLl >). 


Initial element set Clustering for n Clustering for T 2 Clustering for ra 



In this problem, it is necessary to examine a set of change trajectories for each element. As a result, 
multi-stage restructuring problem has to be based on multiple choice model. Generally, this problem is 
very prospective. 


4.2.3. Restructuring in sorting 

One-stage restructuring for sorting problem can be considered as well. Let A = {Ax ,..., A^,..., A„} be 
a initial element set. Solution is a result of dividing set {A} into k linear ordered subsets (ranking): R = 
{i?i, ..., Rj ,..., Rk}, Rj Q AVj = 1, k, \Rj^kRj^ \ = 0 Vji, j2- Linear order is: Ri —>■ ... ^ Rj ... — >■ Rk, 
Ail ^*2 if ^*2 ^ Ji < J2- 

Generally, the sorting problem (or multicriteria ranking) consists in transformation of set A into ranking 
R: A R while taking into account multicriteria estimates of elements and/or expert judgment (e.g., 
[5181666] !. In Fig. 4.9, illustration for restructuring in sorting problem is depicted. The problem is: 

min(5(i?^, i?*) s.t. a{R^ —>■ R*) < b, 

where R* is solution, R^ is initial (the “first”) ranking, R^ is the “second” ranking, 5{R*^ R?) is proximity 
between solution R* and the “second” ranking R* (e.g., structural proximity or proximity by quality 
parameters for rankings), a{R^ —>• R*) is the cost of transformation of the “first” ranking R^ into 
solution R* (e.g., editing “distance”), b is constraint for the transformation cost. Evidently, multi-stage 
restructuring problems (with change trajectories of elements) are prospective as well. 
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Fig. 4.9. Example: restructuring in sorting problem 
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4.3. Clustering with multi-type elements 
4.3.1. Basic problems 

Our basic clustering problem can be considered as the follows: initial set of elements A consists of 
several subsets: A = Uj=i where A^ = {a\, }, j corresponds to a certain kind of element 

type, i.e., there is a set of the types: J = {1,j, ...jt} ■ Special binary relation is defined over the set 
J: Rj. 

The first clustering strategy is: 

(i) to group the initial set of elements A without analysis of the element types, 

(ii) to examine a connection of the elements of different types at the next stage (e.g., for each obtained 
cluster). 

Here, the second clustering strategy is examined: 

(1) to obtain clustering for each subset A^ = {A \,..., A\ , A^j.,}. 

(2) to find a correspondence between clusters of subset A^^ and clusters of subset A^^ (for the case 


(3) to find a correspondence between cluster elements of subsets A^ and cluster elements of subsets 
A^'^ (for the case R{ 3 ^,j'^) = !)■ 

Here, three new clustering problems with multi-type elements are suggested and examined: (i) cluster¬ 
ing with two-type elements, (ii) clustering with three-type elements, and (iii) clustering with four-type 
elements. Examples of the special binary relation(s) over the element types are depicted in Fig. 4.10. 
Fig. 4.11 and Fig. 4.12 illustrate the above-mentioned second clustering strategy (three-type elements): 

(i) clustering of subsets: 

G = G'iUG 2UG3 where Gi = G 2 = {gl,g 2 ,gh, G 3 = {51,32:ffll; 

B = B,\JB 2 \JB 3 where Hi = {&{,&!, bl}, B 2 = {bl bl b^}, B 3 = {6?, &i, &i}; 

H = Hi\JH2\JH3 where Hi = {h\,h^,hl}, H2 = H3 = {hf, hf, h|}; 

(ii) three-matching of the obtained clusters: < Hi*G3*i72 >, < B 2 *Gi*H 3 >, and < B 3 -kG 2 *Hi >. 

(iii) three-matching of cluster elements: 

< Bl * G3 ★ H2 >: < ★ 5 i * ^3 ^ ^2 * 33 * ^2 ^ ^3 * 32 * ^1 

< i?2 -k Gi k H 3 >: b\k g^ k , K b'^k g\ k , K b^k g 2 k 

< B3 k G2 k Hi >: < b\k g^k h\ >, < ★ 31 * ^1 ^ * 33 * ^2 


1 o 

2 ' 

3 (i) 

4 @ 

(a) two types (b) three types (c) four types 

Fig. 4.10. Illustration for binary relation over element types 











Fig. 4.11. Clustering strategy (three-type elements): clustering, three-matching 


Problem solving frameworks are based on combinations of well-known combinatorial problems and cor- 
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responding algorithms: clustering (e.g., hierarchical clustering, k-means clustering), assignment/allocation, 
three-matching, for example: 

(a) clustering of elements for each element subset, 

(b) assignment of the obtained clusters (while taking into account the binary relation over element 
types), 

(c) assignment of cluster elements (while taking into account the binary relation over element types), 
and 

(d) analysis of the obtained solution and its correction/improvement. 
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Fig. 4.12. Three- 
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4.3.2. Example of Team Design 

The problems of analysis, modeling and design of teams are widely used in many domains (R&D, 
Start-Up companies, manufacturing, education, etc.). Some basic types of teams are briefly described in 
Table 4.4. 


Table 4.4. Types of teams 


No. 

Type 

Purpose(s) 

Domain(s) 

Source(s) 

1 . 

R&D project team 

Accomplishment of specific task 

R&D 

12041 

2 . 

Multi-functional / 
multi-disciplinary 
team 

Forming of multi-functional/ 
multi-disciplinary task(s) 

(system life cycle) 

R&D, design, 
manufacturing, 
logistics, marketing, 
investment, etc. 

112116511 

3. 

Formal work group 

To deliver a product or service 

R&D, design, 
manufacturing, 
logistics, etc. 

[5S5] 

4. 

Global virtual 

teams 

Forming and management of 
distributed team(s) 

Integrated 

distributed 

technologies 

|403I497| 

5. 

Informal group 
(e.g., friends) 

To collect and pass on business 
information 

Business 


6 . 

Community of 
practice 

(group of experts) 

Building and sharing/exchange 
of knowledge, coordination 

Professional 

networks, 

organizations 

|37114781612| 

7. 

Start-Up team 

Design of new product/service 

Innovation 


8 . 

Student team 

Joint educational work, 
joint research project 

Education 

this paper 

9. 

Professors/lecturers 

teams 

Design of/participation in 
new educational program 

Education 



Some additional description for two special types of teams are: 

(a) “communities of practice” are the groups of experts in applied domains (with interaction, exchange 
of knowledge and experience, etc.), 

(b) global virtual (distributed) teams involves four major elements: (i) virtual team structure, (ii) 
strategic objectives, (iii) work characteristics, (iv) situational constraints [497] . 

The discipline of teams |322) contains several basic research problems, for example: 
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(1) design of teams (e.g., structure, elements) (e.g., [39214031497] 1. 

(2) selection/forming of multi-functional teams (e.g., [3921651] !. 

(3) analysis of relationships between team design and team performance (e.g., [559] !. 

(4) modeling of teams evolution and forecasting (e.g., [392] !. 

Example 4.3. A simplified numerical example for designing a multi-student teams for joint labora¬ 
tory/project works is described. General solving framework consists of seven stages (Fig. 4.13): 

Stage 1. Preliminary data analysis (expert judgment). 

Stage 2. Analysis and generation/processing of criteria (expert judgment, databases). 

Stage 3. Assessment of objects upon criteria, processing of estimates (expert judgment, computing). 

Stage 4- Grouping/classification of initial object set to obtain object subset for each type (hltering, 
classification/sorting). 

Stage 5. Evaluation of relationship between object pairs (e.g., “friendship”/compatibility) (expert 
judgment, databases). 

Stage 6. Design of object configurations (special composite structures) (e.g., morphological clique 
problem). 

Stage 7. Analysis of results. 




Team 

struc¬ 

tures 

(^Object subset 1 

) 


(^Object subset 2 


Conh- 


gura- 

(^Object subset {k - 

TJ)- 

ting 

(^Object subset k 


teams 


(Xeam l) 
( ^Team ^ 

(^Team 


Fig. 4.13. Solving framework 


Here example for 14 students (from Table 2.2) is considered: Table 4.5. The considered basic set of 
criteria (educational disciplines) is the following (ordinal scale [3,4,5], 5 corresponds to the best level): 

(1) mathematics Ci, (2) physics C 2 , (3) computer systems Ca, (4) software engineering C 4 , (5) antenna 
devices C 5 , ( 6 ) signal processing Cq, (7) receiver and sender systems C 7 , ( 8 ) information transmission 
Cg, (9) measurement in radio engineering Cg, (10) control systems Ciq. 

The following student types are examined (by professional inclination/skill): 

(1) formal methods (Oi), support disciplines by criteria: Ci, Cg, Cg, Cg, Cio; 

(2) physical processes (O 2 ), support disciplines by criteria: C 2 ,Cg-, 

(3) system design (O 3 ), support disciplines by criteria: C 3 ,Ce,Cs,Cio; 

( 4 ) software development (O4), support disciplines by criteria: C3,C4; 

(5) simulation (O 5 ), support disciplines by criteria: Ci,C 3 ,Ci] 

( 6 ) signal processing (Og), support disciplines by criteria: Ci, C 3 , ( 74 , Cg, Cg; 

(7) data transmission (O 7 ), support disciplines by criteria: ( 71 , ( 72 , (^g. Cl's, (^g, ( 77 , (7g. 

Thus, it is necessary to transform initial estimates of students upon basic set of criteria (i.e., (7i,..., Cg) 
into estimates by inclination (i.e., (7i, (72, (73, ( 74 ) with selection of “domain leader”/“quasi-domain 
leader” for each inclination type (i.e., by some rule(s), multicritria ranking/sorting problem, expert 
judgment) (Table 4.6). Here, 1 corresponds to level of “domain leader”, 2 corresponds to level of “quasi¬ 
domain leader”. 

In the example, the problem purpose is to form the following laboratory/project student teams (joint 
laboratory works, student joint projects): 

(1) monitoring system project Ti, support professional skills (as team structure): Ti = Oi * O 3 * Oi* 
O5 * Ot, 

( 2 ) medical systems (measurement and analysis) T 2 , support professional skills (as team structure): 
T 2 = Oi * O 3 -k O 4 * (7g; 
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(3) GIS and seismic modeling T 3 , support professional skills (as team structure): T 3 = Oi *02 *03 * 
O 4 * O 5 * Og. 

Ordinal estimates of student friendship is presented in Table 4.7 (expert judgment, ordinal scale 
[0,1, 2,3] is used, 3) corresponds to the best friendship). 

The design process is based on morphological clique problem while taking into account ordinal quality 
of elements (Table 4.6) and elements compatibility (Table 4.7). Structures for configuration design of 
team Ti, team T 2 , team T 3 are depicted in Fig. 4.14, in Fig. 4.15, and in Fig. 4.16. 

It is assumed that each student team consists of 2, 3, or 4 students. “Domain leader(s)” or “quasi¬ 
domain leader(s)” have to be contained into each team. In educational process, elements which are not 
“domain leaders” / “quasi-domain leaders” have to be added into some some student teams (composite 
solutions). Numerical examples of composite solutions are: 

(a) for Ti (Fig. 4.14): 

= Tig * Tig * A2 * TI33 , = tIq * tIq * TI2 * TI33, = Tig * Tig * A2 , = Tig * Aq * Tig ] 

(b) for Tg (Fig. 4.15): 

Xg = Aq * Tig * Tig * ^n, = Tig * Tig * Tig * TI33, Xj^ = Tig * Tig * ^2, X^ = Tig * Tig * TlgJ 

(c) for X 3 (Fig. 4.16): 

= Tig * Tig * Tig * ^n, = Tig * Tig * Tig * TI33, = Tig * Tig * Tig, = Tig * Tig * Tig. 


Table 4.5. Students, estimates upon criteria 


Item (student) 

Cl 

Gg 

G3 

G4 

Gg 

Gg 

Gt 

Gs 

Gg 

Gio 

Student 1 

3 

3 

4 

4 

4 

4 

5 

5 

5 

4 

Student 2 

3 

3 

3 

3 

3 

4 

4 

3 

4 

3 

Student 3 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

Student 4 

5 

5 

4 

4 

4 

4 

4 

5 

3 

4 

Student 5 

3 

3 

3 

4 

3 

4 

4 

3 

4 

3 

Student 6 

5 

4 

5 

4 

5 

5 

5 

5 

5 

4 

Student 7 

3 

3 

3 

4 

3 

4 

4 

3 

4 

3 

Student 8 

4 

4 

4 

4 

3 

4 

4 

4 

5 

4 

Student 9 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

Student 10 

5 

5 

5 

4 

5 

5 

5 

5 

5 

5 

Student 11 

3 

3 

3 

3 

3 

3 

4 

3 

5 

3 

Student 12 

3 

4 

4 

4 

4 

4 

4 

4 

5 

4 

Student 13 

5 

5 

5 

5 

5 

4 

4 

5 

4 

5 

Student 14 

3 

4 

4 

4 

4 

4 

4 

4 

4 

4 


Table 4.6. Students, evaluation by inclination types 


Item (student) 

Oi 

Gg 

O3 

O4 

O5 

Gg 

O 7 

Student I 
Student 2 
Student 3 

2 



2 


2 

2 

Student 4 
Student 5 

2 

2 




2 

2 

Student 6 

1 

2 

1 

2 

2 

1 

1 

Student 7 
Student 8 


2 

2 



2 


Student 9 

I 

1 

1 

1 

1 

1 

1 

Student 10 

1 

1 

1 

2 

2 

1 

1 

Student II 
Student 12 


2 

2 

2 




Student 13 

1 

2 

1 

1 

1 

1 

2 

Student 14 




2 
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Table 4.7. Ordinal estimates of student friendship (compatibility) 


Discipline 

A 2 

^3 

A 4 

A 5 

Aq 

A7 

^8 

A 9 

Aio 

All 

Ai 2 

^13 

Ai4 

Ai 
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3 

1 

3 

3 

1 

1 

2 

2 

3 

3 

1 

1 

A 2 


1 

2 

1 

2 

3 

2 

1 

1 

1 

2 

1 

2 

A 3 



3 

3 

3 

1 

1 

2 

3 

3 

3 

1 

1 

A 4 




2 

3 

1 

1 

2 

1 

3 

3 

0 

2 

A 5 





3 

1 

1 

3 

2 

2 

3 

1 

0 

Aq 






0 

0 

1 

0 

3 

3 

1 

1 

A 7 







3 

2 

3 

1 

2 

1 

2 

As 








3 

3 

1 

2 

1 

0 

Ag 









3 

2 

3 

2 

1 

Aio 










3 

3 

2 

0 

All 











3 

2 

2 

Ai 2 












2 

2 

^13 
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Fig. 4.15. 4-component team T 2 


# T3 — Oi * O2 k O3 * O4 k O5 k Oq 
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Fig. 4.16. 6-component team T 3 

Note, the system problem to design a trajectory of composite teams (or multistage team design) is 
very prospective one [392] . 

4.3.3. Analysis of Network 

Multi-type clustering strategy can be applied for analysis in networks. The types of networks nodes can 
be obtained by analysis of their connections (the number and types of neighbors): (a) multi-connected 
nodes (type 1), (b) connected nodes (type 2), (c) outliers (type 3), (d) isolated nodes (type 4). 
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Let G = {A,E) be the examined network (graph), where A = {Ai,Ai,An} is the set of nodes, 
E is the set of edges {\E\ = h). The following clustering scheme can be considered: 

Stage 1. Building the list of nodes (with info on neighbors) 0{n) 

Stage 2. Selection of multi-neighbor nodes (type 1) (complexity estimate equals 0{n)). Result: subset 
Bi c A. 

Stage 3. Selection of outlier nodes (i.e., leaves, type 3) (complexity estimate equals 0{n)). Result: 
subset d {A Bi}. 

Stage 4- Selection of other nodes (type 2) (complexity estimate equals 0(n)). Result: subset i?2 C A, 
B 2 = {A {R1US3}. 

Stage 5. Clustering of multi-neighbor nodes Bi (complexity estimate equals about 0(|i?ip) (about 
0((n/3)^)). Thus, a preliminary clustering solution is: = {X}, , ...,X}^^}. Now, a macro-network 

can be built: = {X^,E^), where is the node set that corresponds to the obtained clustering solution 

(i.e., set of clusters), E^ is a built set of edges. (Note, the obtained clusters can be used as centroids in 
k-means clustering at the next step(s)). 

Stage 6 . Clustering of nodes of type 2, i.e., set B 2 (if needed). The corresponding clustering solution 
is: X"^ = {Xf ,..., Xf ,..., X^^}. Now, a macro-network can be built: G^ = {X‘^,E‘^), where X"^ is the node 
set that corresponds to the obtained clustering solution (i.e., set of clusters), E"^ is a built set of edges. 

Stage 7. Matching of two graphs G^ = {X^,E^) and G^ = {X‘^,E‘^). The matching process can be 
based on edges from initial network or on the usage of additional parameters (e.g., node coordinates). As 
a result, integrated clustering solution can be obtained X^^ = {X}^, ...,Xl‘^, ...,X}^‘^^}. 

Stage 8 . Joining outliers {B 3 ) to clusters of solution X^^. As a result, integrated clustering solution 
can be obtained 

4.4. Scheduling in multi-beam antenna communication system 

There are the following initial information (Fig. 4.17): 

(a) multi-beam antenna system, 

(b) number of antenna beams: /r, 

(c) set of communication nodes A = {Ai ,..., Ai ,..., A„}, 

(d) volume of transmitted data is about the same for each Ai. 

The problem is: 

Design a schedule for connection of antenna system to communication nodes while taking into account 
the following: (i) minimization of total connection time, (ii) providing the best communication quality 
(by the minimum interference between neighbor (by angle) connections, i.e., 

max min (A,,, 

i^A 

where (A^^, Ai^) is angular separation (A^^, A^^) (section 2.2.4) (or angle between beams 

to the nodes). 

The solving scheme is the following: 

Stage 1. Linear ordering of communication nodes by their angle (Fig. 4.17, node 1 is the 1st). 

Stage 2. Dividing of the obtain list of nodes into /i equal by size groups (the last group can have less 
elements) and numeration as follows: 

group 1: {(l,l),(l,2),...,(l,fc)}, 

group 2: {(2,1), (2, 2),..., (2, fc)}, 

groups: {(^, 1),(^,2),...,(^,A:)}. 

AeTek=\^}. _ 

Stage 3. Generation of scheduling by the rules: Slot 7 (7 = l,fc): the 7-th element from each group 
(C = l,2...7r), i.e., elements {(,j)} (Fig. 4 . 17 ). 

Stage 4- Stop. 
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Node 1 Node i Node n 



Fig. 4.17. Multi-beam antenna system 
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5. Conclusions 

The paper addresses combinatorial modular viewpoints to clustering problems and procedures: (a) 
generalized modular clustering frameworks (i.e., typical combinatorial engineering frameworks); (b) main 
structural clustering models/methods (e.g., hierarchical clustering, minimum spanning tree based clus¬ 
tering, clustering as assignment, detection of clisue/quasi-clique based clustering, correlation clustering, 
network communities based clustering); (c) main ideas for fast clustering schemes. Described problem 
solving frameworks are based on compositions of well-known combinatorial optimization problems and 
corresponding algorithms (e.g., assignment, partitioning, assignment, knapsack problem, multiple choice 
problem, morphological clique problem, searching for consensus/median for structures). 

A set of numerical examples illustrate all stages of clustering processes (problem statement, assessment 
and evaluation, design of solving algorithms/schemes, analysis of results). 

Future research directions can involve the following: 

1. analysis and design of new composite problems and modular solving frameworks; 

2. design of special decision support tools (modular solving environments) for structural clustering 
problems; 

3. special study of dynamic structural clustering problems and and very large applications; 
applications of structural clustering problems in networking (design, covering, routing, etc.); and 

5. application of the structural clustering problems in education and in educational data mining. 
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